cronicl: data processing pipelines.
cronicl is a simple data processing pipeline tool written in Python.
No additional infrastructure needed, if you can run Python, you can process data through cronicl.
Pipelines can be short-lived (e.g. to process a file) or long-lived (e.g. to process streaming input).
Features
- Programatically define pipeline flow and processing steps
- Record the number of times each step is executed and execution time
- Automatic version tracking of processing steps
- Trace messages through the pipeline (random samples)
- Monitor via HTTP interface
