PipelineX OverviewΒΆ
PipelineX is a Python package to build ML pipelines for experimentation with Kedro, MLflow, and more
PipelineX provides the following options which can be used independently or together.
HatchDict: Python in YAML/JSON
HatchDict
is a Python dict parser that enables you to include Python objects in YAML/JSON files.Note:
HatchDict
can be used with or without Kedro.Flex-Kedro: Kedro plugin for flexible config
Flex-Kedro-Pipeline: Kedro plugin for quicker pipeline set up
Flex-Kedro-Context: Kedro plugin for YAML lovers
MLflow-on-Kedro: Kedro plugin for MLflow users
MLflow-on-Kedro
provides integration of Kedro with MLflow with Kedro DataSets and Hooks.Note: You do not need to install MLflow if you do not use.
Kedro-Extras: Kedro plugin to use various Python packages
Kedro-Extras
provides Kedro DataSets, decorators, and wrappers to use various Python packages such as:Note: You do not need to install Python packages you do not use.
Please refer here to find out how PipelineX differs from other pipeline/workflow packages: Airflow, Luigi, Gokart, Metaflow, and Kedro.