airflow-provider-bigquery-reservation

An Apache Airflow provider concerning BigQuery reservation.


Keywords
airflow, bigquery, reservation
License
Apache-2.0
Install
pip install airflow-provider-bigquery-reservation==1.1.1

Documentation

Airflow BigQuery Pricing

Airflow BigQuery Reservation Provider


PyPI version codecov Github Action Test

Warning This package is a pre-released of the official apache-airflow-providers-google package. All of these operators will be integrated to the official package, soon.

This repository provides an Apache Airflow provider based on BigQuery Reservation API.

Airflow Operators

  • BigQueryReservationCreateOperator: Buy BigQuery slots (commitments) and assign them to a GCP project (reserve and assign).
  • BigQueryReservationDeleteOperator: Delete BigQuery commitments and remove associated ressources (rservation and assignment).
  • BigQueryBiEngineReservationCreateOperator: Create or Update a BI engine reservation.
  • BigQueryBiEngineReservationDeleteOperator: Delete or Update a BI engine reservation.

You could find DAG samples here.

Requirements

  • A Google Cloud connection has to be defined. By default, all hooks and operators use google_cloud_default.
  • This connection requires the following roles on the Google Cloud project(s) used in these operators:

Defining a new dedicated connection and custom GCP role could be good practices to respect the principle of least privilege.

How to install

pip install --user airflow-provider-bigquery-reservation