MySQL I/O Connector of Apache Beam


Keywords
apache beam mysql connector
License
MIT
Install
pip install beam-mysql-connector==1.7.4

Documentation

Beam - MySQL Connector

PyPI version PyPI - Python Version Codacy Badge License: MIT

Beam - MySQL Connector is an io connector of Apache Beam to access MySQL databases.

Installation

pip install beam-mysql-connector

Getting Started

  • Read From MySQL
from beam_mysql.connector import splitters
from beam_mysql.connector.io import ReadFromMySQL


read_from_mysql = ReadFromMySQL(
        query="SELECT * FROM test_db.tests;",
        host="localhost",
        database="test_db",
        user="test",
        password="test",
        port=3306,
        splitter=splitters.DefaultSplitter(),  # you can select how to split datasets
)
  • Write To MySQL
from beam_mysql.connector.io import WriteToMySQL


write_to_mysql = WriteToMySQL(
        host="localhost",
        database="test_db",
        table="tests",
        user="test",
        password="test",
        port=3306,
        batch_size=1000,
)

License

MIT License. Please refer to the LICENSE.txt, for further details.