Notice: we’re no longer actively developing Orchest. Please read more here.
🙌
Build data pipelines, the easy way No frameworks. No YAML. Just write your data processing code directly in Python, R or Julia.
Note: Orchest is in beta.
Features
- Visually construct pipelines through our user-friendly UI
- Code in Notebooks and scripts (quickstart)
- Run any subset of a pipelines directly or periodically (jobs)
- Easily define your dependencies to run on any machine (environments)
- Spin up services whose lifetime spans across the entire pipeline run (services)
- Version your projects using git (projects)
When to use Orchest? Read it in the docs.
Roadmap
Missing a feature? Have a look at our public roadmap to see what the team is working on in the short and medium term. Still missing it? Please let us know by opening an issue!
Examples
Get started with an example project:
- Train and compare 3 regression models
- Connecting to an external database using SQLAlchemy
- Run dbt in Orchest for a dbt + Python transform pipeline
- Use PySpark in Orchest
Installation
Want to skip the installation and jump right in? Then try out our managed service: Orchest Cloud.
Slack Community
Join our Slack to chat about Orchest, ask questions, and share tips.
License
The software in this repository is licensed as follows:
- All content residing under the
orchest-sdk/
andorchest-cli/
directories of this repository are licensed under theApache-2.0
license as defined inorchest-sdk/LICENSE
andorchest-cli/LICENSE
respectively. - Content outside of the above mentioned directories is available under the
AGPL-3.0
license.
Contributing
Contributions are more than welcome! Please see our contributor guides for more details.
Alternatively, you can submit your pipeline to the curated list of Orchest
examples that are automatically loaded in every
Orchest deployment!