beamism

beamism is a tiny framework to run apache beam based software.

Motivation

• Implement Mongo Sink
• Schema Validation
• Flow Control

Installation

pip install beamism

Quick Start(Examples)

# run locally
python examples/wordcount_streaming.py

# deploy to Google Cloud Dataflow
python examples/wordcount_streaming.py --runner DataflowRunner --project {YOUR_PROJECT}--setup_file {ABSOLUTE_PATH_TO_SETUP.PY} --requirements_file {ABSOLUTE_PATH_TO_REQUIREMENTS.TXT} --temp_location {GCP_STORAGE_PATH} --staging_location {GCP_STORAGE_PATH}

※ This module is inspired by Apache Beam, and some codes are modified based on it.