Dataflow programming for python


Keywords
dataflow, machine-learning, python
License
Apache-2.0
Install
pip install pythonflow==0.3.0

Documentation

Pythonflow: Dataflow programming for python. Build Status Documentation Status

Pythonflow is a simple implementation of dataflow programming for python. Users of Tensorflow will immediately be familiar with the syntax.

At Spotify, we use Pythonflow in data preprocessing pipelines for machine learning models because

  • it automatically caches computationally expensive operations,
  • any part of the computational graph can be easily evaluated for debugging purposes,
  • it allows us to distribute data preprocessing across multiple machines.

See the documentation for details.