distify

Wrapper around Ray for easy distributed processing in Python.

Disclaimer: I use it for some of my projects, but there is no guarantee that the code will work as expected or that future versions will be backward compatible.

Install

pip install distify --upgrade

Features

• Multiple backends: Ray, Multiprocessing, Multithreading, Sequential.
• Logging.
• Progress bar.
• Can run in local or in multiple nodes.
• Individual timeout for map applications.
• Resume from checkpointing.
• Hydra integration.

Quickstart/usage

Please refer to this example: https://github.com/jordiae/distify/tree/main/examples/basic

Documentation

TODO

LICENSE

MIT

Author

Jordi Armengol Estapé