A Framework for Enabling Scientific Workflow Research and Development

This Python package provides a collection of tools for:

• Analyzing instances of actual workflow executions;
• Producing recipes structures for creating workflow recipes for workflow generation;
• Generating synthetic realistic workflow instances; and
• Generating realistic workflow benchmark specifications.

WfCommons is available on PyPI. WfCommons requires Python3.8+ and has been tested on Linux and macOS.

While pip can be used to install WfCommons, we suggest the following approach for reliable installation when many Python environments are available:

$ python3 -m pip install wfcommons

If you want to use the latest WfCommons unstable version, that will contain brand new features (but also contain bugs as the stabilization work is still underway), you may consider retrieving the latest unstable version.

Cloning from WfCommons's GitHub repository:

$ git clone https://github.com/wfcommons/wfcommons
$ cd wfcommons
$ pip install .

WfCommons uses pygraphviz for generating visualizations for the workflow task graph. If you want to enable this feature, you will have to install the graphviz package (version 2.16 or later). You can install graphviz easily on Linux with your favorite package manager, for example for Debian-based distributions:

sudo apt-get install graphviz libgraphviz-dev

and for RedHat-based distributions:

sudo yum install python-devel graphviz-devel

On macOS you can use brew package manager:

brew install graphviz

Then you can install pygraphviz by running:

python3 -m pip install pygraphviz

WfCommons uses pydot for reading and writing DOT files. If you want to enable this feature, you will have to install the pydot package:

python3 -m pip install pydot

The main channel to reach the WfCommons team is via the support email: support@wfcommons.org.

Bug Report / Feature Request: our preferred channel to report a bug or request a feature is via
WfCommons's Github Issues Track.

When citing WfCommons, please use the following paper. You should also actually read that paper, as it provides a recent and general overview on the framework.

@article{wfcommons,
    title = {{WfCommons: A Framework for Enabling Scientific Workflow Research and Development}},
    author = {Coleman, Taina and Casanova, Henri and Pottier, Loic and Kaushik, Manav and Deelman, Ewa and Ferreira da Silva, Rafael},
    journal = {Future Generation Computer Systems},
    volume = {128},
    number = {},
    pages = {16--27},
    doi = {10.1016/j.future.2021.09.043},
    year = {2022},
}