abstractions package

Submodules

abstractions.abs_exceptions module

abstractions.augmentation module

abstractions.base_class module

class abstractions.base_class.BaseClass(config: Optional[abstractions.utils.ConfigStruct] = None)[source]

Bases: abc.ABC

abstractions.data_loading module

abstractions.evaluation module

abstractions.model_building module

abstractions.orchestration module

abstractions.preprocessing module

abstractions.submit_training_job module

abstractions.training module

abstractions.utils module

class abstractions.utils.ConfigStruct(**entries)[source]

Bases: object

Structure for loading config as a Python object.

seed
Type

int

input_height

for resizing and model creation, this is input-height of your model’s input and preprocessor’s output

Type

int

input_width

for resizing and model creation, this is input-width of your model’s input and preprocessor’s output

Type

int

src_code_path

relative to project(repository)_dir,

Type

str

data_dir

required for testing, provide absolute path to dataset, your data-loader should work using this path. you can provide a different dataset directory when submitting a training job.

Type

str

data_loader_class

Required, relative to src_code_path

Type

str

model_builder_class

Required, relative to src_code_path

Type

str

preprocessor_class

Required, relative to src_code_path

Type

str

augmentor_class

relative to src_code_path

Type

str

evaluator_class

Required, relative to src_code_path

Type

str

epochs
Type

int

batch_size
Type

int

data_loader

parameters for instantiating DataLoader

Type

Struct

model_builder

parameters for instantiating ModelBuilder

Type

Struct

preprocessor

parameters for instantiating Preprocessor

Type

Struct

augmentor

parameters for instantiating Augmentor

Type

Struct

do_train_augmentation
Type

bool

do_validation_augmentation
Type

bool

export

parameters for exporting, will be used by trainer

Type

Struct

project_name
Type

str

class abstractions.utils.Struct(**entries)[source]

Bases: object

abstractions.utils.add_config_file_to_mlflow(config_dict: dict)[source]

Adds parameters from config file to mlflow.

Parameters

config_dict – config file as a nested dictionary

abstractions.utils.check_for_config_file(run_dir: pathlib.Path) → pathlib.Path[source]

Checks for existence of config file and returns the path to config file if exists.

Raises

  • ConfigNotFound –

  • FoundMultipleConfigs –

Returns

path to config file

abstractions.utils.get_logger(name: str) → logging.Logger[source]
abstractions.utils.load_config_as_dict(path: pathlib.Path) → dict[source]

loads the yaml config file and returns a dictionary

Parameters

path – path to json config file

Returns

a nested object in which parameters are accessible using dot notations, for example config.model.optimizer.lr

abstractions.utils.load_config_file(path: pathlib.Path) → abstractions.utils.ConfigStruct[source]

loads the yaml config file and returns a ConfigStruct

Parameters

path – path to json config file

Returns

a nested object in which parameters are accessible using dot notations, for example config.model.optimizer.lr

abstractions.utils.setup_mlflow(mlflow_tracking_uri, mlflow_experiment_name: str, base_dir: pathlib.Path, evaluation=False) → mlflow.tracking.fluent.ActiveRun[source]

Sets up mlflow and returns an active_run object.

tracking_uri/
experiment_id/

run1 run2 …

Parameters

  • mlflow_tracking_uri – tracking_uri for mlflow

  • mlflow_experiment_name – experiment_name for mlflow, use the same experiment_name for all experiments

  • use. (related to the same task. This is different from the experiment concept that we) –

  • base_dir – directory for your experiment, containing your config.yaml file.

  • evaluation – if evaluation==true, then new run will be created, named base_dir.name + _evaluation

Returns

an active_run object to use for mlflow logging.

Return type

active_run

Module contents