github.com/arunsworld/functional


License
Apache-2.0
Install
go get github.com/arunsworld/functional

Documentation

Functional programming style APIs in Go - with concurrency support

Concepts

  • Pipeline - allows for the definition of a data pipeline with a sequence of PreFilters, Transformers and PostFilters
  • PipelineComponents - utility structure to define the pipeline components. Including concurrency.
  • FilterOperation - allows determination of whether an element should stay or be filtered out during pipeline processing
  • TransformOperation - transforms element. additionally supports filtering. additionally supports returning errors
  • FoldOperation - folds / reduces elements down

Pipeline

The Pipeline is the overarching concept that provides the executable behviours of:

  • Apply - takes a slice and applies the data pipeline to it
  • ApplyAndFold - applies data pipelien to slice and then folds it down to a single element
  • Stream - continuously streams data through the pipeline until closure

Concurrency

  • Set the Concurrency in PipelineComponents to greater than 1 to get transparent concurrency when processing the data pipeline.
  • Data is smartly distributed between independent jobs that execute the data pipeline across the filters, transforms and even the folding operation.
  • During streaming this may naturally result in out of sequence data handling, so beware!
  • During slice processing (Apply) the result is sorted by original sequence allowing sequence to be maintained.
  • Goes without saying that when using concurrency all functions must be thread-safe.