https://github.com/fablabbcn/smartcitizen-data
A python package for analyzing environmental sensor's data
https://github.com/fablabbcn/smartcitizen-data
air-quality data-analysis data-science environmental-monitoring low-cost-sensor machine-learning python sensor-data sensors smartcitizen
Last synced: about 16 hours ago
JSON representation
A python package for analyzing environmental sensor's data
- Host: GitHub
- URL: https://github.com/fablabbcn/smartcitizen-data
- Owner: fablabbcn
- License: gpl-3.0
- Created: 2017-07-19T19:07:36.000Z (almost 9 years ago)
- Default Branch: master
- Last Pushed: 2026-05-21T16:59:32.000Z (23 days ago)
- Last Synced: 2026-05-29T14:02:52.468Z (15 days ago)
- Topics: air-quality, data-analysis, data-science, environmental-monitoring, low-cost-sensor, machine-learning, python, sensor-data, sensors, smartcitizen
- Language: Python
- Homepage: https://docs.smartcitizen.me/data/data-tools/
- Size: 104 MB
- Stars: 17
- Watchers: 17
- Forks: 10
- Open Issues: 9
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- open-sustainable-technology - Smart Citizen Data - A python package for analyzing environmental sensor's data. (Natural Resources / Air Quality)
README
Smart Citizen Data
=======
[](https://zenodo.org/badge/latestdoi/97752018)
[](https://mybinder.org/v2/gh/fablabbcn/smartcitizen-data-framework/master?filepath=%2Fexamples%2Fnotebooks)
[](https://badge.fury.io/py/scdata)
[](https://github.com/fablabbcn/smartcitizen-data/actions/workflows/python-multiple-versions.yml)
Welcome to **SmartCitizen Data**. This is a data analysis framework for working with sensor data in different ways:
- Interacting with several sensors APIs
- Clean data, export and calculate metrics
- Model sensor data and calibrate sensors
- Generate data visualisations - matplotlib, ~[plotly](https://plotly.com/)~ or [uplot](https://leeoniya.github.io/uPlot)
- Generate analysis reports in html or pdf and upload them to [Zenodo](http://zenodo.org)
## Installation
You can check it out in the [](https://mybinder.org/v2/gh/fablabbcn/smartcitizen-data-framework/master?filepath=%2Fexamples%2Fnotebooks) before installing if you want. Works with `Python 3.*` (tested between 3.9 and 3.11).
You can just run:
```
pip install scdata
```
### Work on the source code
Simply clone the repository with:
```
git clone https://github.com/fablabbcn/smartcitizen-data.git
cd smartcitizen-data
```
Install `scdata` package with requirements:
```
python setup.py install
```
Or if you want to edit:
```
cd scdata
pip install --editable .
```
### Tokens and config
A configuration file is available at `~/.config/scdata/config.yaml`, which contains a set of configurable variables to allow or not the local storage of relevant data in the data folder, normally in `~/.cache/scdata`:
```
data:
cached_data_margin: 2
load_cached_api: true
reload_metadata: true
store_cached_api: true
paths:
config: /Users/username/.config/scdata
data: /Users/username/.cache/scdata
export: /Users/username/.cache/scdata/export
interim: /Users/username/.cache/scdata/interim
inventory: ''
models: /Users/username/.cache/scdata/models
processed: /Users/username/.cache/scdata/processed
raw: /Users/username/.cache/scdata/raw
reports: /Users/username/.cache/scdata/reports
uploads: /Users/username/.cache/scdata/uploads
zenodo_real_base_url: https://zenodo.org
zenodo_sandbox_base_url: http://sandbox.zenodo.org
```
Also, `.env` files will be picked from `~/.cache/scdata`. An [env.example](env.example) is provided. If you want to upload data to [Zenodo](http://zenodo.org), you will need to fill set an environment variable called `ZENODO_TOKEN` in your environment. The `.env` file is not mandatory for `scdata` to work. It only serves for authentication if you want to `POST` or `GET` data without throttling. You can still do `GET` requests but they will be throttled if not authenticated/authorized by the platform. See [user roles](https://developer.smartcitizen.me/#authentication) for more info.
### Using with jupyter lab (optional)
It can also be used with `jupyter lab` or `jupyter`. For this [install juypterlab](https://github.com/jupyterlab/jupyterlab).
## Contribute
Issues and PR more than welcome!
## Funding
This work has received funding from the European Union's Horizon 2020 research and innovation program under the grant agreement [No. 689954](https://cordis.europa.eu/project/rcn/202639_en.html)