https://github.com/rupeshtr78/mqttspark

IOT Device MQTT Spark Streaming
https://github.com/rupeshtr78/mqttspark

cassandra gpio iot mqtt mqtt-broker mqtt-client raspberry-pi spark spark-streaming yarn

Last synced: 2 days ago
JSON representation

IOT Device MQTT Spark Streaming

• Host: GitHub
• URL: https://github.com/rupeshtr78/mqttspark
• Owner: rupeshtr78
• Created: 2020-05-09T18:43:47.000Z (over 4 years ago)
• Default Branch: master
• Last Pushed: 2022-10-05T19:47:26.000Z (about 2 years ago)
• Last Synced: 2023-04-27T05:36:28.414Z (over 1 year ago)
• Topics: cassandra, gpio, iot, mqtt, mqtt-broker, mqtt-client, raspberry-pi, spark, spark-streaming, yarn
• Language: Scala
• Size: 15.6 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 4
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
We will create a  IOT Device / MQTT Client using Raspberry Pi .With a simple push button switch to simulate an event using bread board connected to the Pi device.

In the event of pressing button the device will publish access logs ( In our case we will read from a file line by line) to the broker. Later we will create a Spark Streaming application which will read the published topic and related messages into spark stream .Using Spark we will parse the log data ,count the status field using Spark window functions and watermark. Finally  we will write the stream of data to Cassandra sink.

More details of this implementation can be found here on my [blog](https://rupeshtr78.github.io/blog/)