https://github.com/miltiadiss/ceid_ne4348-big-data-management-systems

This project implements a real-time data pipeline with Kafka, Spark, and MongoDB. It generates vehicle data using UXSIM, streams it to a Kafka broker, processes it with Spark, and stores raw and processed data in MongoDB. Queries analyze vehicle counts, speeds, and routes over specified periods.

kafka-consumer kafka-producer pymongo pysaprk spark-sql uxsim

Last synced: 30 Mar 2025