{"id":40538890,"url":"https://github.com/sylwesterf/pms3003","last_synced_at":"2026-01-20T22:51:50.559Z","repository":{"id":89243751,"uuid":"101987461","full_name":"sylwesterf/pms3003","owner":"sylwesterf","description":"Air quality monitoring station ","archived":false,"fork":false,"pushed_at":"2023-02-15T23:11:07.000Z","size":1704,"stargazers_count":4,"open_issues_count":1,"forks_count":4,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-01-30T15:46:29.936Z","etag":null,"topics":["air-quality","air-quality-monitor","air-quality-sensor","dash","dynamodb","ec2","flask","python","r","raspberry-pi","visualization"],"latest_commit_sha":null,"homepage":"https://sylwester.cf/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sylwesterf.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2017-08-31T10:03:56.000Z","updated_at":"2023-03-23T03:29:21.000Z","dependencies_parsed_at":null,"dependency_job_id":"c1a58db7-4eae-4a99-8688-fbc25b2e99a2","html_url":"https://github.com/sylwesterf/pms3003","commit_stats":{"total_commits":435,"total_committers":2,"mean_commits":217.5,"dds":0.08275862068965523,"last_synced_commit":"5d7dddfb00f8d59a56542a66bb26fdd0801e9a3b"},"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/sylwesterf/pms3003","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sylwesterf%2Fpms3003","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sylwesterf%2Fpms3003/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sylwesterf%2Fpms3003/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sylwesterf%2Fpms3003/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sylwesterf","download_url":"https://codeload.github.com/sylwesterf/pms3003/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sylwesterf%2Fpms3003/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28617734,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-20T22:24:05.405Z","status":"ssl_error","status_checked_at":"2026-01-20T22:20:31.342Z","response_time":117,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["air-quality","air-quality-monitor","air-quality-sensor","dash","dynamodb","ec2","flask","python","r","raspberry-pi","visualization"],"created_at":"2026-01-20T22:51:49.998Z","updated_at":"2026-01-20T22:51:50.544Z","avatar_url":"https://github.com/sylwesterf.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Air quality monitoring station \n![lang](https://img.shields.io/github/languages/top/sylwesterf/pms3003.svg?style=flat)\n![size](https://img.shields.io/github/repo-size/sylwesterf/pms3003.svg?style=flat)\n![lastdt](https://img.shields.io/github/last-commit/sylwesterf/pms3003.svg?style=flat)\n![rls](https://img.shields.io/github/release-date-pre/sylwesterf/pms3003.svg?style=flat)\n![lic](https://img.shields.io/github/license/sylwesterf/pms3003.svg)\n\nThis project consists of three parts:\n1. Air quality monitoring station based of Raspberry Pi Zero W and PMS3003 sensor (+DHT22 for temperature and humidity, +BMP180 for pressure)\n2. Data transfer and storage (MySQL/DynamoDB/S3/MongoDB/Kafka) \n3. Data visualization (Plotly/Chart.js/R Shiny/Flask hosted on S3/EC2/github.io)\n\nhttp://smog.cf/\u003c/br\u003e\nhttp://sylwesterf.s3-website.eu-central-1.amazonaws.com/\u003c/br\u003e\nhttps://sylwesterf.github.io/\n\n## Installation\n#### 1. Air quality monitoring station\n\nClone github repository to your Raspberry Pi Zero W and install dependencies.\n\n```sh\n# download project files\nsudo git clone https://github.com/sylwesterf/pms3003.git\ncd pms3003\n\n# install dependencies\nsudo pip3 install -r requirements.txt\n```\n\nFollow RaspberryPi documentation to enable uart: https://www.raspberrypi.org/documentation/configuration/uart.md\n\nConnect PMS3003 to Raspberry Pi as per sensor datasheet:\n\n| PMS3003       | Rpi           |\n|           --- |---            |\n| VCC           | +5V           |\n| GND           | GND           |\n| RxD           | TxD           |\n| TxD           | RxD           |\n\n\nConnect DHT22 (3 PIN) to Raspberry Pi as per sensor datasheet:\n\n| DHT22         | Rpi           |\n|           --- |---            |\n| VCC           | +3.3V         |\n| GND           | GND           |\n| OUT           | GPIO7 (BCM)   |\n\nTo add BMP180 you need to enable i2c first and install as per https://github.com/adafruit/Adafruit_Python_BMP. Connect as per sensor datasheet:\n\n| BMP180        | Rpi           |\n|           --- |---            |\n| VCC           | +3.3V         |\n| GND           | GND           |\n| SDA           | SDA           |\n| SCL           | SCL           |\n\nTest the set up. Make sure you specify a correct path to gpio serial port for device variable (line 9) and select appropriate value for environment variable (line 10) - 0/1 for a sensor placed outdoor/indoor respectively. Use *serial-test.py* (scripts folder) to troubleshoot issues with serial port configuration.\n\n```sh\n# run a test - output to terminal\nsudo python test.py\n```\n\n#### 2. Data transfer and storage\n\n##### DynamoDB\n\nIn order to use AWS DynamoDB as a storage option you need to set up programmatic access for your Raspberry Pi (use AWS CLI) and edit *rpi2dynamodb.py* file (modify variables accordingly):\n- path - a project path used for backup csv generation (line 11)\n- environment - sensor environment:  0/1 for a sensor placed outdoor/indoor respectively (line 14)\n- device - gpio serial port (line 18)\n- dynamodb_table - DynamoDB table to write data to (line 22)\n\nNote: AWS IAM user should have write priviliges for DynamoDB and S3.\n\n```sh\n# run aws configure and set AWS Access Key ID and AWS Secret Access Key for DynamoDB/S3 upload\nsudo aws configure\n\n# run rpi2dynamodb.py script to load data into DynamoDB and generate a (backup) csv file on Raspberry Pi\nsudo python3 rpi2dynamodb.py\n#sudo python3 rpi2dynamodb_onlypm.py\n```\n\nA *csv2s3.py* file can be used to upload CSVs (generated by *rpi2dynamodb.py*) into S3. Two variables need to be updated: \n- s3bucket - AWS S3 bucket name (line 8)\n- filename - CVS filename inculding its path (line 9)\n\n```sh\n# automatic archival of csv files into S3\nsudo python csv2s3.py\n```\n\nUpon successful testing a cronjob can be set up:\n- to measure the air quality hourly and sent data into DynamoDB table,\n- to archive csv files into S3 at the end of the day.\n\n##### MySQL\n\nTo store air quality data in MySQL a fct_pm table needs to be created first and necessary permissions set (update mysql user password):\n\n```sql\nCREATE TABLE db_pms3003.fct_pm\n(\npm1 int NOT NULL,\npm25 int NOT NULL,\npm10 int NOT NULL,\ndt datetime NOT NULL\n);\n\nCREATE USER 'rpi' IDENTIFIED BY 'xxx';\nGRANT INSERT ON db_pms3003.fct_pm TO 'rpi';\n```\n\nInstall packages on Raspberry Pi:\n```sh\npip install mysql-connector-python\n```\n\nUpdate the following in *rpi2mysql.py* file:\n- environment - sensor environment:  0/1 for a sensor placed outdoor/indoor respectively (line 9)\n- device - gpio serial port (line 14)\n- cnx_string - host and password for MySQL (line 17)\n\n##### Apache Kafka\n\nInstall packages\n```sh\npip install kafka-python\n```\n\nUpdate the following in *rpi2kafka.py* file:\n- environment - sensor environment:  0/1 for a sensor placed outdoor/indoor respectively (line 11)\n- device - gpio serial port (line 16)\n- kafka_server (line 19)\n- kafka_username (line 20)\n- kafka_password (line 21)\n- topic - kafka topic to write data to (line 24)\n\n*rpi2kafka.py* is configured to send data (in an infinite loop) every 5 seconds.\n\n##### MongoDB\n\nInstall packages\n```sh\npip install pymongo\n```\n\nUpdate the following in *rpi2mongodb.py* file:\n- environment - sensor environment:  0/1 for a sensor placed outdoor/indoor respectively (line 11)\n- device - gpio serial port (line 16)\n- mongo_server (line 19)\n- mongo_db (line 20)\n- mongo_col (line 21)\n\n*rpi2mongodb.py* is configured to send data (in an infinite loop) every 1 second.\n\n#### 3. Data visualization \n\n##### Flask - NEW (hosted on AWS EC2) \nFlask application assumes a DynamoDB table is created and populated using a solution described in DynamoDB section under 2. Data transfer and storage. \u003c/br\u003e\nAttach an IAM role to EC2 for DynamoDB Read.\u003c/br\u003e\nAdd commands below to EC2 user data when launching an instance or ssh into it and run it afterwards.\u003c/br\u003e\nUpdate variables AWS_REGION and DYNAMODB_TABLE with your AWS region and DynamoDB table name (see sample).\u003c/br\u003e\nYou can also set DASH_USR and DASH_PWD variables that are going to be used to authenticate into one of the app routes - /all \u003c/br\u003e\n\n```sh\n#!/bin/bash\nexport AWS_REGION=your_aws_region\nexport DYNAMODB_TABLE=your_dynamodb_table\n\n# sample\n#export AWS_REGION=eu-central-1\n#export DYNAMODB_TABLE=pms3003\n\nexport DASH_USR=test_usr\nexport DASH_PWD=test_pwd\n\n# prep script\nsudo curl https://raw.githubusercontent.com/sylwesterf/pms3003/master/viz/py-new/prep.sh -o prep.sh\nsudo bash prep.sh $AWS_REGION $DYNAMODB_TABLE $DASH_USR $DASH_PWD\nsudo rm prep.sh\n```\n\nFlask app directory:\n```sh\n/opt/pms3003/\n.\n├── pms3003.py\n├── latest.py\n├── all.py\n├── dockerfile\n├── fun.py\n├── assets/\n│   └── favicon.ico\n├── file.py\n├── wsgi.py\n├── prep.sh\n├── output.csv\n└── requirements.txt\n```\n\n##### Flask - NEW (non AWS)\nFlask application assumes a DynamoDB table is created and populated using a solution described in DynamoDB section under 2. Data transfer and storage. \u003c/br\u003e\nIf you want to use a compute service from a different provider than AWS an IAM user with programmatic access needs to be created and then configured on a destination server (install awscli and run aws configure).\u003c/br\u003e\nThe same set of commands (as for EC2) can be used for deployment; ssh into an instance, update variables (AWS_REGION, DYNAMODB_TABLE, DASH_USR, DASH_PWD) and run a prep.sh script.\n\n##### Flask - Docker\nFlask application assumes a DynamoDB table is created and populated using a solution described in DynamoDB section under 2. Data transfer and storage. \u003c/br\u003e\nPull an image from DockerHub repo and run a container with setting environment variables for AWS Region, DynamoDB table name (default is *'pms3003'*) and number of days to display on graph (default is 21).\n\n```sh\n# install docker\nsudo yum install docker -y\nsudo systemctl start docker\n\n# pull from dockerhub repo\nsudo docker pull sylwesterf/pms3003:latest\nsudo docker images -a\n\n# run a container (attach an IAM role to EC2 for DynamoDB Read)\nsudo docker run -p 80:8000 -e AWS_REGION=\"xyz\" -e DYNAMODB_TABLE=\"xyz\" -e DT_FILTER=21 sylwesterf/pms3003:latest\n#sudo docker run -p 80:8000 -e AWS_REGION=\"eu-central-1\" -e DYNAMODB_TABLE=\"pms3003\" -e DT_FILTER=6 sylwesterf/pms3003:latest\n```\n\nYou can run the same container without an IAM role assumed by EC2 Instace, but by utilizing an IAM user with programmatic access and read permissions for DynamoDB. Set AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables by providing them in docker's run command.\n\n```sh\n# run a container without assumed IAM role\nsudo docker run -p 80:8000 -e AWS_ACCESS_KEY_ID=\"xyz\" -e AWS_SECRET_ACCESS_KEY=\"xyz\" -e DASH_USR=\"xyz\" -e DASH_PWD=\"xyz\" -e AWS_REGION=\"xyz\" -e DYNAMODB_TABLE=\"xyz\" DT_FILTER=21 sylwesterf/pms3003:latest\n```\n\nA dockerfile used to build the image is available under *viz\\py\\dockerfile*.\nFollow below commands to create an image on your own by pulling and building from *viz/py*. folder from this repo.\n```sh\nsudo docker build https://github.com/sylwesterf/pms3003.git#master:viz/py --tag pms3003\nsudo docker run -p 80:8000 -e AWS_REGION=\"xyz\" -e DYNAMODB_TABLE=\"xyz\" -e DT_FILTER=21 pms3003\n```\n\n##### Flask - OLD (hosted on AWS EC2) \nFlask application assumes a DynamoDB table is created and populated using a solution described in DynamoDB section under 2. Data transfer and storage. \u003c/br\u003e\nAttach an IAM role to EC2 for DynamoDB Read.\u003c/br\u003e\nAdd commands below to EC2 user data when launching an instance or ssh into it and run it afterwards.\u003c/br\u003e\nUpdate variables AWS_REGION and DYNAMODB_TABLE with your AWS region and DynamoDB table name (see sample).\u003c/br\u003e\n\n```sh\n#!/bin/bash\nexport AWS_REGION=your_aws_region\nexport DYNAMODB_TABLE=your_dynamodb_table\n\n# sample\n#export AWS_REGION=eu-central-1\n#export DYNAMODB_TABLE=pms3003\n\n# prep script\nsudo curl https://raw.githubusercontent.com/sylwesterf/pms3003/master/viz/py/prep.sh -o prep.sh\nsudo bash prep.sh $AWS_REGION $DYNAMODB_TABLE\nsudo rm prep.sh\n```\n\nFlask app directory:\n```sh\n/opt/pms3003/\n.\n├── fun.py\n├── assets/\n│   └── favicon.ico\n├── wsgi.py\n└── requirements.txt\n```\n\n##### R-Shiny\nShiny application assumes a MySQL table was created and populated using a solution described in MySQL section under 2. Data transfer and storage. \u003c/br\u003e\nDeploy the contents of *pms3003/viz/R/pms3003/* into your shiny server \u003c/br\u003e\nFor shiny server setup on EC2 follow the *pms3003/viz/R/pms3003/ec2_ubuntu_config_R.sh* \u003c/br\u003e\nCreate a 'shiny' user on MySQL server (update mysql user password).\n\n```sql\nCREATE USER 'shiny' IDENTIFIED BY 'xxx';\nGRANT SELECT ON db_pms3003.fct_pm TO 'shiny';\n```\n\n##### Chart.js (hosted on github.io)\nDeploy the contents of *pms3003/viz/js-chart/* into your github.io webpage. Update the url for json src file in *script.js* \u003c/br\u003e\nMake sure to enable CORS for S3 bucket as per: https://stackoverflow.com/questions/49493279/react-js-how-to-get-rid-of-cross-origin-error-in-codesandbox if you're sourcing a json file from S3 \u003c/br\u003e\nSee https://sylwesterf.github.io/ and corresponding repo https://github.com/sylwesterf/sylwesterf.github.io\n\nYou can use AWS CLI to dump DynamoDB table into json file.\n```sh\naws dynamodb scan --table-name TABLE_NAME \u003e output.json\n```\n##### plotly.js (hosted on AWS S3) \nUpload the contents of *pms3003/viz/js-plotly/* into your S3 bucket and update the url for json src file in *script.js* \u003c/br\u003e\n\nYou can clone this github repo, update the url for json src file in *script.js* and run below AWS CLI command from *pms3003/viz/js-plotly* directory (update your bucket name) to deploy the visualization:\n```sh\naws s3 cp . s3://your_bucket_name/ --recursive\n```\n\nSee http://sylwesterf.s3-website.eu-central-1.amazonaws.com\n\n#### 4. Extras\n\nRefer to *pms3003/scripts/* for helpful scripts: \u003c/br\u003e\n- dynamodb-update-table.py (add humidity and temperature data to DynamoDB table)\n- dynamodb2json.py (export data to json format)\n- index-reset-reload.py (move data from non-indexed to indexed table)\n- manual-load-from-csv.py (perfrom a manual load from csv to DynamoDB)\n- power-consumption.sh (reduce RasberryPi W power consumption)\n- serial-test.py (test gpio port)\n- rpi2dynamodb_onlypm.py (excludes dht11,bmp180 data)\n\n## TODO\n- ~~add temperature and humidity sensor~~\n- ~~a javascript viz~~\n- ~~add PM2.5=25 limit threshold line~~\n- ~~new layout~~\n- case for sensors \n- ~~authentication~~\n- ~~sending data to kafka~~\n- ~~sending data to mongodb~~\n- ~~contenerize viz~~\n- ~~python 3~~\n- ~~lightweight distro for container~~ (idea abandoned as per: https://pythonspeed.com/articles/alpine-docker-python/)\n- add barometer\n- ~~cloud provider agnostic deployment~~\n\n## Acknowledgments\n- https://github.com/Thomas-Tsai/pms3003-g3\n- https://medium.com/@rfreeman/serverless-dynamic-real-time-dashboard-with-aws-dynamodb-a1a7f8d3bc01\n- https://github.com/szazo/DHT11_Python\n- https://github.com/okomarov/dash_on_flask\n- https://medium.com/@kmmanoj/deploying-a-scalable-flask-app-using-gunicorn-and-nginx-in-docker-part-1-3344f13c9649\n- https://github.com/adafruit/Adafruit_Python_BMP\n\n## License\nThis project is licensed under the MIT License - see the [LICENSE.md](LICENSE.md) file for details\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsylwesterf%2Fpms3003","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsylwesterf%2Fpms3003","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsylwesterf%2Fpms3003/lists"}