chariots

chariots aims to be a complete framework to build and deploy versioned machine learning pipelines.

• Documentation: https://chariots.readthedocs.io.

Getting Started: 30 seconds to Chariots:

You can check the :doc:`documentation<https://chariots.readthedocs.io>` for a complete tutorial on getting started with chariots, but here are the essentials:

you can create operations to execute steps in your pipeline:

>>> from chariots.sklearn import SKUnsupervisedOp, SKSupervisedOp
>>> from chariots.versioning import VersionType, VersionedFieldDict, VersionedField
>>> from sklearn.decomposition import PCA
>>> from sklearn.linear_model import LogisticRegression
...
...
>>> class PCAOp(SKUnsupervisedOp):
...     training_update_version = VersionType.MAJOR
...     model_parameters = VersionedFieldDict(VersionType.MAJOR, {"n_components": 2})
...     model_class = VersionedField(PCA, VersionType.MAJOR)
...
>>> class LogisticOp(SKSupervisedOp):
...     training_update_version = VersionType.PATCH
...     model_class = LogisticRegression

Once your ops are created, you can create your various training and prediction pipelines:

.. testsetup::

    >>> from chariots._helpers.doc_utils import IrisFullDataSet

>>> from chariots import Pipeline, MLMode
>>> from chariots.nodes import Node
...
...
>>> train = Pipeline([
...     Node(IrisFullDataSet(), output_nodes=["x", "y"]),
...     Node(PCAOp(MLMode.FIT_PREDICT), input_nodes=["x"], output_nodes="x_transformed"),
...     Node(LogisticOp(MLMode.FIT), input_nodes=["x_transformed", "y"])
... ], 'train')
...
>>> pred = Pipeline([
...     Node(PCAOp(MLMode.PREDICT), input_nodes=["__pipeline_input__"], output_nodes="x_transformed"),
...     Node(LogisticOp(MLMode.PREDICT), input_nodes=["x_transformed"], output_nodes=['__pipeline_output__'])
... ], 'pred')

Once all your pipelines have been created, deploying them is as easy as creating a creating a Chariots object:

.. testsetup::
    >>> import tempfile
    >>> import shutil

    >>> app_path = tempfile.mkdtemp()

>>> from chariots import Chariots
...
...
>>> app = Chariots([train, pred], app_path, import_name='iris_app')

The Chariots class inherits from the Flask class so you can deploy this the same way you would any :doc:`flask application<https://github.com/pallets/flask>`.

Once this the server is started, you can use the chariots client to query your machine learning micro-service from python:

>>> from chariots import Client
...
...
>>> client = Client()

.. testsetup::

    >>> from chariots import TestClient
    ...
    ...
    >>> client = TestClient(app)

with this client we will be

• training the models
• saving them and reloading the prediction pipeline (so that it uses the latest/trained version of our models)
• query some prediction

>>> client.call_pipeline(train)
>>> client.save_pipeline(train)
>>> client.load_pipeline(pred)
>>> client.call_pipeline(pred, [[1, 2, 3, 4]])
[1]

.. testsetup::

    >>> shutil.rmtree(app_path)

Features

• versionable individual op
• easy pipeline building
• easy pipelines deployment
• ML utils (implementation of ops for most popular ML libraries with adequate Versionedfield) for sklearn and keras at first
• A CookieCutter template to properly structure your Chariots project

Comming Soon

Some key features of Chariot are still in development and should be coming soon:

• Cloud integration (integration with cloud services to fetch and load models from)
• Graphql API to store and load information on different ops and pipelines (performance monitoring, ...)
• ABTesting

Credits

This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template. audreyr/cookiecutter-pypackage's project is also the basis of the Chariiots project template