Strome

Strome (from Danish Stromen /ˈstroː.mə(n)/, stream) is a Python micro-framework for building processing pipelines.

Concept

The main entity Strome operates with is "PipelineElement". A list of Pipeline Elements with parameters is called "Pipeline".

Pipeline Element represents a piece of logic performing some transformation on given input. Pipeline element could read input from initial set of files as well as output of previous pipeline elements. It also could register one or more outputs to make it available for pipeline elements located after the current one.

Pipeline element optionally could have a set of parameters which allows user to tune behaviour.

Example of the pipeline defined in YAML (taken from pandoc-flow, the document pre-processor based on Strome):

pipeline:
  - name: easyeda.bom.LoadBOM
    params:
      bom: 'modules/OutputModule/bom.csv'
  - name: bom-lcsc
  - name: jinja2
  - pandoc

In this example user defined 4 pipeline elements. They will be invoked sequentially to generate the final document:

• Step 1. Bill of materials will be generated by easyeda.bom.LoadBOM
• Step 2. Bill of materials will be augmented with extra data from internet by bom-lcsc. This pipeline element reads input from the previous processor
• Step 3. Jinja2 template processor will be applied to resolve substitutions in template files
• Step 4. All content generated by previous steps will be passed to Pandoc to generate pdf document.

Built-in pipeline elements

Jinja2

Renders all input files as Jinja2 templates.

Dependencies (libs): jinja2
Provides: jinja2-output-files, List[str] - list of paths pointing generated files Requires: none Parameters:

• output-dir, str - path to the folder where generated files will be placed, default: <tmp-dir>/jinja2
• search-path, List[str] - additional directories added to jinja2 search path, default: []

External components

TBD

Extending classpath

TBD

Loading external components

TBD

Projects using Strome

• Pandoc Flow, pre-processor for Pandoc

Development environment

Ensure you have pre-requisites installed:

• Python 3.7+
• Make

Run make setup - This will create VirtualEnv for project and install required dev. dependencies Before making any commit make use to run make to apply formatter, run linter and update copyright.

Credits

• Dmitry Berezovsky, author
• CLI Rack

Disclaimer

This module is licensed under MIT. This means you are free to use it in commercial projects.

The MIT license clearly explains that there is no warranty for this free software. Please see the included LICENSE file for details.