Smart Citizen Data

Welcome to SmartCitizen Data. This is a data analysis framework for working with sensor data in different ways:

Interacting with several sensors APIs
Clean data, export and calculate metrics
Model sensor data and calibrate sensors
Generate data visualisations - matplotlib, plotly or uplot
Generate analysis reports in html or pdf and upload them to Zenodo

A full documentation of the framework is detailed in the Smart Citizen Docs.

Installation

You can check it out in the before installing if you want. Works with Python 3.* (tested until python 3.9.5).

You can just run:

pip install scdata

Work on the source code

Simply clone the repository with:

git clone https://github.com/fablabbcn/smartcitizen-data.git
cd smartcitizen-data

Install scdata package with requirements:

python setup.py install

Or if you want to edit:

cd scdata
pip install --editable .

Tokens and config

If you want to upload data to Zenodo, you will need to fill set an environment variable called ZENODO_TOKEN in your environment. You can get more instructions here and with this example.

A configuration file is available at ~/.config/scdata/config.yaml, which contains a set of configurable variables to allow or not the local storage of relevant data in the data folder, normally in ~/.cache/scdata:

data:
  cached_data_margin: 2
  load_cached_api: true
  reload_metadata: true
  store_cached_api: true
paths:
  config: /Users/username/.config/scdata
  data: /Users/username/.cache/scdata
  export: /Users/username/.cache/scdata/export
  interim: /Users/username/.cache/scdata/interim
  inventory: ''
  models: /Users/username/.cache/scdata/models
  processed: /Users/username/.cache/scdata/processed
  raw: /Users/username/.cache/scdata/raw
  reports: /Users/username/.cache/scdata/reports
  uploads: /Users/username/.cache/scdata/uploads
zenodo_real_base_url: https://zenodo.org
zenodo_sandbox_base_url: http://sandbox.zenodo.org

Also, .env files will be picked from ~/.cache/scdata.