kedroio
A module extending the datasets that come shipped with kedro
Example usage
-- example.sql
select *
from "database"."table_name"
limit 5;# conf/base/catalog.py
my_athena_dataset:
type: kedroio.datasets.aws.athena.AthenaQueryDataSet
filepath: data/01_raw/example.csv
sql_filepath: example.sql
bucket: example-bucket
workgroup: primary
subfolder: data
region_name: eu-west-2
read_result: true # read into pandas DataFrame
overwrite: false # skip download if filepath existsTesting
Start moto server for mocked AWS resources
moto_serverRun tests
pytest tests/
