Check input and output to conditionally execute commands in parallel
Single job, will only run if any output does not exist, try -n for a dry run:
$ iox -i input1 input2 -o output1 output2 --exec "command {input} {output}"
Single job rerun if inputs are newer than outputs:
$ iox -u -i input1 input2 -o output1 output2 --exec "command {input} {output}"
Parallel jobs:
$ iox -i input1/{d}.csv -o output/{d}.csv -d 2020 2021 2022 \\
--exec "command {input} {output} && echo {d} > {output}"
Parallel jobs with wildcards from a file:
$ iox -i input1/{f}.csv -o output/{f}.csv --date file.txt \\
--exec "echo {date} > {output}"
Parallel jobs with combinations of wildcards:
$ iox -i input_{a}/{d}.csv -o output/{a}_{d}.csv \\
--combinations -a type1 type2 type2 -d 2020 2021 2022 \\
--exec "command --year {d} -t {a} {input} {output}"
Jobs in a pipeline, input/output paths are passed through:
$ echo f1 f2 f3 \
| iox -o f4 -x "echo {input} > {output}" \
| iox -o summary -x "cat {input} > {output}"
vim iox.py # update version string
git commit # make sure workdir is clean
./iox.py -o venv -x virtualenv {output}
source venv/bin/activate
pip install build twine pytest
pytest tests.py
./iox.py -i *.py -o dist -u -x python -m build
python -m twine upload dist/*
git tag v$(./iox.py --version)
git push; git push --tags
