Data Mechanics Airflow Plugin
An Airflow plugin to launch and monitor Spark applications on the Data Mechanics platform.
Environment
- Python >= 3.5
- apache-airflow >= 1.10.x
Installation and usage
A tutorial to configure and use this plugin is available in the Data Mechanics docs.
Example DAGs
You can see example DAGs in here.