Ceci Pipeline Software
“Ceci n'est pas une pipeline.”
A lightweight parsl-based framework for running DESC pipelines.
This is now beta status.
pip install ceci
This installs the simplest version of ceci, if you want to be able
to use the parsl backend, install instead
You can then run an example pipeline from the ceci_lib directory using:
export PYTHONPATH=$PYTHONPATH:$PWD ceci test/test.yml
Adding Pipeline Stages
To make new pipeline stages, you must:
- make a new python package somewhere else, to contain your stages.
- the package must have an
__init__.pyfile that should import from
.all the stages you want to use.
- it must also have a file
__main__.pywith the same contents as the example in
- each stage is its own class inheriting from
ceci.PipelineStage. Each must define its name, inputs, and outputs, and a run method.
- the run method should use the parent methods from
PipelineStageto get its inputs and outputs etc.