bqpipe

Wrapper around BigQuery & Snowflake libraries to simplify writing to/reading from Pandas DataFrames.


Keywords
bigquery, snowflake, google, cloud, etl, data, engineering, dataframe, pandas
License
OpenSSL
Install
pip install bqpipe==0.5.2

Documentation

bqpipe: Warehouse & DataFrame

A lightweight Python wrapper around the Google Cloud BigQuery and Snowflake data platform APIs to simplify read and write between Pandas DataFrames and BigQuery/Snowflake.

Features

  • Create new tables with specified schema
  • Append DataFrames to existing tables
  • Simply read from tables or arbitrary SELECT statements
  • Get BigQuery metadata (datasets, tables, table schemas, etc.)

Dependencies

BQPipe supports Python 3.7+.

Installation requires NumPy, Pandas, Google-Cloud-BigQuery, Snowflake-Connector-Python, Snowflake-SQLAlchemy, and Cryptography.

Installation

The latest stable release (and older versions) can be installed from PyPI:

pip install bqpipe

You may instead want to use the development version from Github:

pip install git+https://github.com/4mile/bqpipe.git#egg=bqpipe

Testing

Coming Soon

Development

BQPipe development takes place on GitHub.

Please submit any reproducible bugs you encounter to the Issue Tracker.