pipelinex.extras.decorators package

Submodules

pipelinex.extras.decorators.decorators module

pipelinex.extras.decorators.decorators.log_time(func)[source]

A function decorator which logs the time taken for executing a function.

Parameters

func (Callable) – The function to be logged.

Return type

Callable

Returns

A wrapped function, which will execute the provided function and log the running time.

pipelinex.extras.decorators.memory_profiler module

pipelinex.extras.decorators.memory_profiler.mem_profile(func)[source]

A function decorator which profiles the memory used when executing the function. The logged memory is collected by using the memory_profiler python module and includes memory used by children processes. The usage is collected by taking memory snapshots every 100ms. This decorator will only work with functions taking at least 0.5s to execute due to a bug in the memory_profiler python module. For more information about the bug, please see https://github.com/pythonprofilers/memory_profiler/issues/216

Parameters

func (Callable) – The function to be profiled.

Return type

Callable

Returns

A wrapped function, which will execute the provided function and log its max memory usage upon completion.

pipelinex.extras.decorators.mlflow_logger module

pipelinex.extras.decorators.mlflow_logger.mlflow_log_time(func)[source]

A function decorator which logs the time taken for executing a function.

Parameters

func (Callable) – The function to be logged.

Return type

Callable

Returns

A wrapped function, which will execute the provided function and log the running time.

pipelinex.extras.decorators.nvml_profiler module

pipelinex.extras.decorators.nvml_profiler.get_nv_info()[source]
pipelinex.extras.decorators.nvml_profiler.nvml_profile(func)[source]
Return type

Callable

pipelinex.extras.decorators.pandas_decorators module

pipelinex.extras.decorators.pandas_decorators.df_set_index(cols)[source]

decorator with arguments

Return type

Callable

pipelinex.extras.decorators.pandas_decorators.log_df_summary(func)[source]
Return type

Callable

pipelinex.extras.decorators.pandas_decorators.total_seconds_to_datetime(cols, origin='1970-01-01')[source]

decorator with arguments

Return type

Callable