pipelinex.mlflow_on_kedro.transformers.mlflow package¶
Submodules¶
pipelinex.mlflow_on_kedro.transformers.mlflow.mlflow_io_time_logger module¶
-
class
pipelinex.mlflow_on_kedro.transformers.mlflow.mlflow_io_time_logger.
MLflowIOTimeLoggerTransformer
(enable_mlflow=True, metric_name_prefix='_time_to_')[source]¶ Bases:
kedro.io.transformers.AbstractTransformer
Log duration time to load and save each dataset.
-
__init__
(enable_mlflow=True, metric_name_prefix='_time_to_')[source]¶ - Parameters:
enable_mlflow (
bool
) – Enable logging to MLflow.metric_name_prefix (
str
) – Prefix for the metric names. The metric names are metric_name_prefix concatenated with ‘load <data_set_name>’ or ‘save <data_set_name>’
-
load
(data_set_name, load)[source]¶ This method will be deprecated in Kedro 0.18.0 in favour of the Dataset Hooks before_dataset_loaded and after_dataset_loaded.
Wrap the loading of a dataset. Call
load
to get the data from the data set / next transformer.- Parameters:
data_set_name (
str
) – The name of the data set being loaded.load (
Callable
[[],Any
]) – A callback to retrieve the data being loaded from the data set / next transformer.
- Return type:
Any
- Returns:
The loaded data.
-
save
(data_set_name, save, data)[source]¶ This method will be deprecated in Kedro 0.18.0 in favour of the Dataset Hooks before_dataset_saved and after_dataset_saved.
Wrap the saving of a dataset. Call
save
to pass the data to the data set / next transformer.- Parameters:
data_set_name (
str
) – The name of the data set being saved.save (
Callable
[[Any
],None
]) – A callback to pass the data being saved on to the data set / next transformer.data (
Any
) – The data being saved
- Return type:
None
-