{"id":15208865,"url":"https://github.com/marcobendinelli/noise-level-simulation-and-analysis-system","last_synced_at":"2026-01-19T02:32:23.168Z","repository":{"id":236276727,"uuid":"792247577","full_name":"MarcoBendinelli/Noise-Level-Simulation-and-Analysis-System","owner":"MarcoBendinelli","description":"Implementation of a noise level analysis system for a country. Utilizes both sensor data and computer simulations to analyze noise levels, followed by calculation of key metrics","archived":false,"fork":false,"pushed_at":"2024-04-26T11:03:37.000Z","size":1480,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-10T07:44:12.684Z","etag":null,"topics":["apache-spark","contiki-ng","data-ana","iot","mpi","mqtt","node-red"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MarcoBendinelli.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":"support_code/Info_coordinates.txt","governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-04-26T09:21:15.000Z","updated_at":"2024-04-26T11:03:40.000Z","dependencies_parsed_at":null,"dependency_job_id":"6c93ef1c-7a0b-4fff-a34b-a98b37062582","html_url":"https://github.com/MarcoBendinelli/Noise-Level-Simulation-and-Analysis-System","commit_stats":{"total_commits":7,"total_committers":2,"mean_commits":3.5,"dds":0.2857142857142857,"last_synced_commit":"75584972b960c4884ac198da3ac3f40eccb9b42d"},"previous_names":["marcobendinelli/noise-level-simulation-and-analysis-system"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MarcoBendinelli%2FNoise-Level-Simulation-and-Analysis-System","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MarcoBendinelli%2FNoise-Level-Simulation-and-Analysis-System/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MarcoBendinelli%2FNoise-Level-Simulation-and-Analysis-System/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MarcoBendinelli%2FNoise-Level-Simulation-and-Analysis-System/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MarcoBendinelli","download_url":"https://codeload.github.com/MarcoBendinelli/Noise-Level-Simulation-and-Analysis-System/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247262961,"owners_count":20910330,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache-spark","contiki-ng","data-ana","iot","mpi","mqtt","node-red"],"created_at":"2024-09-28T07:02:45.356Z","updated_at":"2026-01-19T02:32:23.162Z","avatar_url":"https://github.com/MarcoBendinelli.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"# System for Simulating and Analyzing Noise Levels :musical_note:\n\nThis project involves designing and implementing a system to analyze the level of noise in a country. The country is divided into regions, some of which have open access to noise data from sensors, while others do not. The system utilizes sensor data in regions where it's available and generates data using computer simulations in regions without sensors.\n\nFor further details, refer to the full report available [here](Report.pdf).\n\n## More Information\n### Sensor Data\nMobile IoT devices, such as smartwatches, are equipped with sensors that measure noise levels every 10 seconds. A sliding window calculates the average of the last six readings, which are then reported to the backend through static IoT devices acting as IPv6 border routers. If the average noise level exceeds a certain threshold, raw readings are reported instead of the average.\n\nThe IoT devices are implemented using **Contiki-NG** due to its lightweight and efficient operating system tailored for low-power wireless devices. Moreover, **MQTT** was chosen as the network protocol for the IoT devices because of its simplicity and suitability for devices with resource constraints. MQTT works efficiently on IoT devices, allowing for reliable communication over networks with limited bandwidth and intermittent connectivity.\n\n### Noise Levels\nIn regions without sensor data, the system relies on computer simulations or virtual software-emulated sensors. Simulations are based on population dynamics, considering noise from people and vehicles. The simulation takes parameters such as the number of people and vehicles, the dimensions of the region, noise levels produced by individuals and vehicles, and the distances affected by their presence.\n\n**MPI** was used to parallelize the computation of a compute-intensive task, namely the simulation of noise levels. MPI, or Message Passing Interface, is a standardized and portable message-passing system designed for parallel computing. It allows for efficient communication and coordination between multiple processes running on different nodes of a distributed system. This parallelization technique is essential for speeding up the simulation process, enabling it to handle large datasets and complex calculations effectively.\n\n### Data Cleaning and Enrichment\nThe backend receives noise level data from real IoT sensors, simulations, and virtual sensors, annotated with geographic areas. Before storage, the backend performs preprocessing:\n1. Discarding invalid/incomplete measurements.\n2. Associating each reading with the name of the nearest point of interest.\n\n**Node-RED** was chosen due to its browser-based flow editor, which simplifies the process of wiring together flows using a wide range of nodes. Each node in Node-RED implements specific functionalities, eliminating the need to implement them manually. This approach significantly reduces development time and complexity, as well as facilitating integration with other systems and services. Additionally, Node-RED's visual interface allows for easy monitoring and debugging of flows, enhancing overall efficiency and productivity.\n\n### Data Analysis\nThe backend periodically computes metrics including:\n1. Hourly, daily, and weekly moving averages of noise levels for each point of interest.\n2. Top 10 points of interest with the highest noise levels in the last hour.\n3. Point of interest with the longest streak of good noise levels.\n\n**Spark** was chosen for its ability to process a large volume of data efficiently. Specifically, Spark SQL was preferred over Spark Streaming for several reasons. Firstly, it allows for the calculation of past metrics and enables the integration of new metrics seamlessly. The Spark SQL module facilitates easy querying of the dataset, making it simpler to manage. Additionally, it's worth noting that Spark SQL is more cost-effective than Spark Streaming. Unlike Spark Streaming, Spark SQL does not need to run constantly; it can be executed only when required, reducing overall expenses.\n\n## Team members\n| Name and Surname | Github |\n:---: | :---: \n| Matteo Beltrante | [@Beltrante](https://github.com/Beltrante) |\n| Marco Bendinelli | [@MarcoBendinelli](https://github.com/MarcoBendinelli) |\n| Simone Berasi | [@SimoneBerasi](https://github.com/SimoneBerasi) |\n\n## Guide\n\n### Node-RED\n#### First Node-RED setup\nFollow the guide to install the Node-RED on your Windows Subsystem (which is supposed to be a Debian-based Subsystem): https://nodered.org/docs/getting-started/raspberrypi\n#### How to open Node-RED\nStart Node-RED:\n\n```shell\nnode-red\n```\nIf by typing http://127.0.0.1:1880/ the webpage does not appear to you, you should find the local ip address of your Windows Subsystem.\n\nIn fact WSL is a virtual environment, so it has its own virtual network interface.\n\nYou can find it just by opening a Windows PowerShell window and by typing:\n\n```shell\nwsl hostname -I\n```\n    \nSo copy the first ip addres that you find and type in your browser: http://copied_ip_address:1880/\n\nIf you want to speed up this operation, follow this guide: https://discourse.nodered.org/t/how-to-run-node-red-on-windows-10-wsl-linux-subsystem/45032\n#### Palettes to install\nnode-red-contrib-watchdirectory\n\n### MQTT\n#### First Mosquitto setup\nIn your Client machine (VM - Ubuntu 16.04 32-bit Contiki NG Middleware Class AY 2021) modify the Mosquitto configuration file:\n```shell\ncd /etc/mosquitto\n```\n```shell\nsudo nano mosquitto.conf\n```\nAnd add the following lines in the bottom of the file:\n```shell\n  connection bridge-01\n  address mqtt.neslab.it:3200\n  topic # out 0\n  topic # in 0\n```\nSave, Exit and Restart your machine.\n\n### How to run the IoT simulation\nMove the folder Iot_implementation inside the contiki-ng-mw-2122\n\nStart Cooja:\n```shell\ncd /tools/cooja\n```\n```shell\nant run\n```\nOpen the simulation:\n\nFile -\u003e Open Simulation -\u003e Browse -\u003e IoT_Cooja_Simulation.csc\n\nBrowse the correct source files\n\nStart the tunslip6:\n```shell\ncd /rpl-border-router\n```\n```shell\nmake TARGET=cooja connect-router-cooja\n```\nStart the simulation\n\n### MPI\n#### Download \n- Download an MPI toolchain to compile and run MPI C programs. For instance, you can use the [Open MPI implementation](https://www.open-mpi.org/).\n- On Linux, Open MPI is typically available in package managers\n- On mac OS, you can install the open-mpi package through [Homebrew](https://brew.sh/)\n- On Windows, you can use WSL\n#### Compile and Run \nYou can compile and run the simulation with the following commands:\n- mpicc ./simulation.c -o simulation -lm\n- mpirun -np 2 ./simulation\n\n### Apache Spark\n#### Download \n- Download Apache Spark pre-built with hadoop from [this page](https://spark.apache.org/downloads.html)\n#### Run\n- compile data_analysis jar with maven\n- start spark master from inside hadoop folder with following line\n```shell\n    ./sbin/start-master.sh -h 127.0.0.1\n```\n- start spark worker from inside hadoop folder with following line\n```shell\n    ./sbin/start-worker.sh spark://127.0.0.1:7077\n```\n- submit work to spark master \n```shell\n    ./bin/spark-submit --class it.polimi.middleware.spark.DataAnalysis pathToJar masterAddress pathToFolderContainingResults.csv\n```\n- (submit example)\n```shell\n     ./bin/spark-submit --class it.middleware.spark.DataAnalysis /mnt/c/Users/m4rc0/Middleware/Project1/Data_Analysis/target/project_1_spark-1.0.jar spark://127.0.0.1:7077 /mnt/c/Users/m4rc0/Middleware/Project1/Back-end/in_out_files/\n```\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarcobendinelli%2Fnoise-level-simulation-and-analysis-system","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmarcobendinelli%2Fnoise-level-simulation-and-analysis-system","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarcobendinelli%2Fnoise-level-simulation-and-analysis-system/lists"}