https://github.com/quantiply/druid-metrics-to-kafka

Sends Druid metrics to a Kafka topic.
https://github.com/quantiply/druid-metrics-to-kafka

Last synced: 3 months ago
JSON representation

Sends Druid metrics to a Kafka topic.

• Host: GitHub
• URL: https://github.com/quantiply/druid-metrics-to-kafka
• Owner: quantiply
• Created: 2015-05-27T22:33:58.000Z (over 9 years ago)
• Default Branch: master
• Last Pushed: 2015-09-07T15:20:09.000Z (about 9 years ago)
• Last Synced: 2024-05-07T18:24:23.106Z (6 months ago)
• Language: Python
• Size: 179 KB
• Stars: 14
• Watchers: 5
• Forks: 9
• Open Issues: 1
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
Druid metrics collector

===

A simple HTTP server that forwards Druid metrics to a Kafka topic.

Druid's HTTP emitter sends a batch of metrics to the HTTP endpoint. They typically look like this :

```

[

{"feed": "metrics", "user2": "Code Cache", "service": "middlemanager", "user1": "nonheap", "timestamp": "2015-04-22T19:31:17.685Z", "metric": "jvm/pool/max", "value": 50331648, "host": "middlemanager:8089"},

{"feed": "metrics", "user2": "Code Cache", "service": "middlemanager", "user1": "nonheap", "timestamp": "2015-04-22T19:31:17.685Z", "metric": "jvm/pool/max", "value": 50331648, "host": "middlemanager:8089"}

.....

]

```

Install

---

```

apt-get update

apt-get install -y python-pip

pip install kafka-python cherrypy docopt

```

Run

---

1. `druid-metrics-collector.py   [--host=] [--port=]`

2. Configure druid to send to the `http://:/metrics` endpoint.

Test

---

```

curl -XPOST localhost:9999/metrics -H "Content-Type: application/json" -d '[{"feed": "metrics", "user2": "Code Cache", "service": "middlemanager", "user1": "nonheap", "timestamp": "2015-04-22T19:31:17.685Z", "metric": "jvm/pool/max", "value": 50331648, "host": "middlemanager:8089"},{"feed": "metrics", "user2": "Code Cache", "service": "middlemanager", "user1": "nonheap", "timestamp": "2015-04-22T19:31:17.685Z", "metric": "jvm/pool/max", "value": 50331648, "host": "middlemanager:8089"}]'

```