https://github.com/ritreshgirdhar/iot-devicemonitoring
Sample application to capture MQTT Events from IOT devices and monitors it through ELK
https://github.com/ritreshgirdhar/iot-devicemonitoring
capture-events elasticsearch elk iot iot-device iot-platform kibana logstash logstash-forwarder logstash-plugin mosquitto mosquitto-pub mqtt mqtt-broker rabbitmq
Last synced: 26 days ago
JSON representation
Sample application to capture MQTT Events from IOT devices and monitors it through ELK
- Host: GitHub
- URL: https://github.com/ritreshgirdhar/iot-devicemonitoring
- Owner: RitreshGirdhar
- Created: 2019-09-19T08:01:54.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2020-05-07T07:45:11.000Z (about 5 years ago)
- Last Synced: 2025-05-04T08:54:12.578Z (27 days ago)
- Topics: capture-events, elasticsearch, elk, iot, iot-device, iot-platform, kibana, logstash, logstash-forwarder, logstash-plugin, mosquitto, mosquitto-pub, mqtt, mqtt-broker, rabbitmq
- Size: 1.9 MB
- Stars: 2
- Watchers: 2
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# IOT Device Monitoring Sample Application
Sample application to capture Events from IOT devices and monitors it through ELK or store it in events-store or some time-series database for further processing.
Pre-requisite
* Basic knowledge of Git + Docker + Mqtt + Rabbit MQ
Here i am taking a very basic example of capturing weather temperature city-wise. You could use it for any another event capturing.
Purpose of this application is to give you enough idea to set up your own message broker & device-monitoring.
Below are the most Popular Internet of Things Protocols and standard communication technologies :
* MQTT
* DDS
* AMQP
* Bluetooth
* Zigbee
In this tutorial i am using MQTT protocol. To capture events from IOT devices we require mqtt broker to consume the generated events and store it for further processing.
I am using RabbitMQ as MQTT broker ([read it](https://dzone.com/articles/top-10-criteria-for-selecting-a-mqtt-broker-1))and to simulate behavior of IOT devices event generation i will use mosquitto's publisher.
Refer [this](https://youtu.be/deG25y_r6OY) to understand basic understanding of RabbitMQ.
Pre-requisites :
1. Basic understanding of RabbitMQ and ELK.
2. Install RabbitMQ Server.
3. Install mosquitto package.
4. Set up ELK on your machine.
Rabbit MQ Configuration.
** Make sure to enable mqtt plugin in rabbitmq using below command:
```
rabbitmq-plugins enable rabbitmq_mqtt
```
1. Creating Queue:
2. Binding exchange with Queue by routing Key :
3. Publising message
4. Message received
Hope you have installed mosquitto package. Now let's try to send some message through mosquitto_pub.
```
mosquitto_pub -h 127.0.0.1 -t weather.mumbai -m "{"temperature":{"min":21,"max":29,"unit":"celsius"},"time":1568881230}" -u guest -P guest -p 1883 -d
```
Let's check weather events queue in Rabbit MQ.
Start logstash with configuration i.e logstash-rabbitmq.conf. It is a basic configuration,which will read from RabbitMQ and dump events into weather index in Elastic Search.
paste below conf in logstash-rabbitmq.conf file
```
input {
rabbitmq {
host => "localhost"
port => 15672
heartbeat => 30
durable => true
exchange => "amq.topic"
exchange_type => "topic"
key => "weather.*"
}
}
output {
elasticsearch {
hosts => "localhost:9200"
index => "weather"
}
stdout {}
}
```
Start logstash
```
logstash -f /logstash-rabbitmq.conf
```
After logstash startup , you will see logstash queue get created and binded with weather* events in RabbitMQ
Let's publish some test messages through RabbitMQ GUI
Now let's check kibana dashboard of weather-index. As you could see in below Kibana search that our weather-mumbai event received by Elastic-Search through logstash.
Now let's test it through MQTT client/IOT Device (in our case its mosquitto_pub)
```
$ mosquitto_pub -h 127.0.0.1 -t weather.mumbai -m '{"temperature":{"min":21,"max":32,"unit":"celsius"},"timestamp":"2019-09-19T18:59:00"}' -u guest -P guest -p 1883 -d
```
