Reusable utilities for working with Glue PySpark jobs

pip install glue-utils

This library does not include pyspark and aws-glue-libs as dependencies as they are already pre-installed in Glue's runtime environment.

To help in developing your Glue jobs locally in your IDE, it is helpful to install pyspark and aws-glue-libs. Unfortunately, aws-glue-libs is not available through PyPI so we can only install it from its git repository.

pip install pyspark==3.3.0
pip install git+https://github.com/awslabs/aws-glue-libs.git@master

To make your local environment as close to Glue's runtime as possible, use the versions specified in this document.

For more details on what you can use this library for, check out the project wiki.