{"id":20316085,"url":"https://github.com/lupyuen/prometheus-the-things-network","last_synced_at":"2025-09-22T22:31:33.539Z","repository":{"id":87767135,"uuid":"417444725","full_name":"lupyuen/prometheus-the-things-network","owner":"lupyuen","description":"Prometheus Configuration for The Things Network (MQTT)","archived":false,"fork":false,"pushed_at":"2021-10-20T15:45:21.000Z","size":21,"stargazers_count":5,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-05T05:24:59.687Z","etag":null,"topics":["grafana","prometheus","thethingsnetwork"],"latest_commit_sha":null,"homepage":"https://lupyuen.github.io/articles/prometheus","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lupyuen.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-10-15T09:35:45.000Z","updated_at":"2025-02-16T22:39:20.000Z","dependencies_parsed_at":null,"dependency_job_id":"f03d109a-b22d-43fa-b3ca-c9e4aa89f633","html_url":"https://github.com/lupyuen/prometheus-the-things-network","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/lupyuen/prometheus-the-things-network","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lupyuen%2Fprometheus-the-things-network","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lupyuen%2Fprometheus-the-things-network/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lupyuen%2Fprometheus-the-things-network/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lupyuen%2Fprometheus-the-things-network/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lupyuen","download_url":"https://codeload.github.com/lupyuen/prometheus-the-things-network/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lupyuen%2Fprometheus-the-things-network/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":276487242,"owners_count":25651132,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-22T02:00:08.972Z","response_time":79,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["grafana","prometheus","thethingsnetwork"],"created_at":"2024-11-14T18:24:14.912Z","updated_at":"2025-09-22T22:31:33.271Z","avatar_url":"https://github.com/lupyuen.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Prometheus and Grafana for The Things Network (MQTT)\n\nRead the article...\n\n-   [\"Monitor IoT Devices in The Things Network with Prometheus and Grafana\"](https://lupyuen.github.io/articles/prometheus)\n\n![Storing The Things Network Sensor Data with Prometheus](https://lupyuen.github.io/images/grafana-flow2.jpg)\n\nWe assume that the Payload Formatter for The Things Network has been configured...\n\n-   [\"CBOR Payload Formatter for The Things Network\"](https://lupyuen.github.io/articles/payload)\n\n# Ingest The Things Network MQTT Messages into Prometheus\n\nTo ingest MQTT Messages from The Things Network into Prometheus...\n\nDownload [`ttn-mqtt.yaml`](ttn-mqtt.yaml) and configure the MQTT Settings and Metrics.\n\nDownload and run `mqtt2prometheus`...\n\n```bash\n## Download mqtt2prometheus\ngo get github.com/hikhvar/mqtt2prometheus\n\n## For macOS and Linux:\ncd $GOPATH/src/github.com/hikhvar/mqtt2prometheus\n\n## For Windows:\ncd %GOPATH%\\src\\github.com\\hikhvar\\mqtt2prometheus\n\n## Build mqtt2prometheus\ngo build ./cmd\n\n## Run mqtt2prometheus\ngo run ./cmd -log-level debug -config ttn-mqtt.yaml\n```\n\nWe should see...\n\n```text\nmqttclient/mqttClient.go:20     \nConnected to MQTT Broker\n\nmqttclient/mqttClient.go:21     \nWill subscribe to topic{\"topic\": \"#\"}\n\nweb/tls_config.go:191           \n{\"level\": \"info\", \"msg\": \"TLS is disabled.\", \"http2\": false}\n\nmetrics/ingest.go:42    \nGot message     \n{\"topic\": \"v3/luppy-application@ttn/devices/eui-YOUR_DEVICE_EUI/up\", \"payload\": \n  {\n    \"end_device_ids\": {\n        \"device_id\": \"eui-YOUR_DEVICE_EUI\",\n        \"application_ids\": {\n            \"application_id\": \"luppy-application\"\n        },\n        \"dev_eui\": \"YOUR_DEVICE_EUI\",\n        \"join_eui\": \"0000000000000000\",\n        \"dev_addr\": \"YOUR_DEVICE_ADDR\"\n    },\n    \"correlation_ids\": [\n        \"as:up:01FJ3TF5MHDHK9ZP17DJRCVNCC\",\n        \"gs:conn:01FJ36D90C9ETZFTFAB2N64JKM\",\n        \"gs:up:host:01FJ36D98KBG2AKNYWE24GAVCD\",\n        \"gs:uplink:01FJ3TF5DYDHZA5CY2YCXD05NG\",\n        \"ns:uplink:01FJ3TF5E0D1HYPBQRRE0DYBB6\",\n        \"rpc:/ttn.lorawan.v3.GsNs/HandleUplink:01FJ3TF5DZG9Q8RNF52TDP2TKX\",\n        \"rpc:/ttn.lorawan.v3.NsAs/HandleUplink:01FJ3TF5MGSAFH6ANWXJK8R520\"\n    ],\n    \"received_at\": \"2021-10-16T05:51:19.698584300Z\",\n    \"uplink_message\": {\n        \"session_key_id\": \"YOUR_SESSION_KEY_ID\",\n        \"f_port\": 2,\n        \"f_cnt\": 18,\n        \"frm_payload\": \"omF0GROZYWwZD6A=\",\n        \"decoded_payload\": {\n            \"l\": 4000,\n            \"t\": 5017\n        },\n        \"rx_metadata\": [\n            {\n                \"gateway_ids\": {\n                    \"gateway_id\": \"YOUR_GATEWAY_ID\",\n                    \"eui\": \"YOUR_GATEWAY_EUI\"\n                },\n                \"time\": \"2021-10-16T05:58:01.085341Z\",\n                \"timestamp\": 3854967529,\n                \"rssi\": -55,\n                \"channel_rssi\": -55,\n                \"snr\": 13.2,\n                \"location\": {\n                    \"latitude\": 1.27125,\n                    \"longitude\": 103.80795,\n                    \"altitude\": 70,\n                    \"source\": \"SOURCE_REGISTRY\"\n                },\n                \"uplink_token\": \"YOUR_UPLINK_TOKEN\",\n                \"channel_index\": 4\n            }\n        ],\n        \"settings\": {\n            \"data_rate\": {\n                \"lora\": {\n                    \"bandwidth\": 125000,\n                    \"spreading_factor\": 10\n                }\n            },\n            \"data_rate_index\": 2,\n            \"coding_rate\": \"4/5\",\n            \"frequency\": \"922600000\",\n            \"timestamp\": 3854967529,\n            \"time\": \"2021-10-16T05:58:01.085341Z\"\n        },\n        \"received_at\": \"2021-10-16T05:51:19.488029108Z\",\n        \"consumed_airtime\": \"0.370688s\",\n        \"network_ids\": {\n            \"net_id\": \"000013\",\n            \"tenant_id\": \"ttn\",\n            \"cluster_id\": \"ttn-au1\"\n        }\n    }\n  }\n}\n```\n\nTo check the ingested metrics...\n\n```bash\ncurl -v http://localhost:9641/metrics\n```\n\nWe should see...\n\n```text\n# HELP l Light Level\n# TYPE l gauge\nl{sensor=\"eui-YOUR_DEVICE_EUI\",\n  sensor_type=\"l\",\n  topic=\"v3/luppy-application@ttn/devices/eui-YOUR_DEVICE_EUI/up\"\n} 4000 1634364863274\n...\n# HELP received_messages received messages per topic and status\n# TYPE received_messages counter\nreceived_messages{status=\"success\",\n  topic=\"v3/luppy-application@ttn/devices/eui-YOUR_DEVICE_EUI/up\"\n} 3\n...\n# HELP t Temperature\n# TYPE t gauge\nt{sensor=\"eui-YOUR_DEVICE_EUI\",\n  sensor_type=\"t\",\n  topic=\"v3/luppy-application@ttn/devices/eui-YOUR_DEVICE_EUI/up\"\n} 5056 1634364863274\n```\n\n# Configure Prometheus\n\nTo add the ingested metrics to Prometheus, edit `prometheus.yml` and add a Scrape Job for `ttn`...\n\n```yaml\n# Scrape configuration containing the endpoints to scrape\nscrape_configs:\n  ...\n\n  # Scrape The Things Network Metrics from MQTT2Prometheus\n  - job_name: \"ttn\"\n\n    # Metrics will be scraped from MQTT2Prometheus\n    # at http://localhost:9641/metrics\n    static_configs:\n      - targets: [\"localhost:9641\"]\n```\n\nTo see the ingested metrics in Prometheus, browse to...\n\nhttp://localhost:9090/\n\nIn the query box enter...\n\n```text\nt\n```\n\nOr...\n\n```text\nl\n```\n\nClick `Execute → Graph`\n\nWe should see a graph of the Temperature or Light Level...\n\n![Prometheus Graph](https://lupyuen.github.io/images/prometheus-metric.png)\n\n# Configure Grafana\n\nTo add the Prometheus metrics to Grafana, add a Prometheus Data Source with the URL...\n\n```text\nhttp://localhost:9090\n```\n\n![Prometheus Data Source for Grafana](https://lupyuen.github.io/images/prometheus-grafana2.png)\n\nAdd a Grafana Panel with the Prometheus Data Source.\n\nSet the Metric to `t` or `l`...\n\n![Prometheus Panel for Grafana](https://lupyuen.github.io/images/prometheus-grafana3.png)\n\nThe Grafana chart appears...\n\n![Prometheus Chart in Grafana](https://lupyuen.github.io/images/prometheus-grafana.png)\n\n# MQTT with TLS\n\nTo connect `mqtt2prometheus` to The Things Network MQTT Server with __TLS Encryption__ (so that the MQTT Password won't be transmitted in the clear)...\n\nEdit this source file...\n\n```text\n$GOPATH/src/github.com/hikhvar/mqtt2prometheus/cmd/mqtt2prometheus.go\n```\n\nAnd modify the `newTLSConfig` function as follows...\n\n```go\nfunc newTLSConfig(cfg config.Config) (*tls.Config, error) {\n    certpool := x509.NewCertPool()\n    if cfg.MQTT.CACert != \"\" {\n        pemCerts, err := ioutil.ReadFile(cfg.MQTT.CACert)\n        if err != nil {\n            return nil, fmt.Errorf(\"failed to load ca_cert file: %w\", err)\n        }\n        certpool.AppendCertsFromPEM(pemCerts)\n    }\n\n    // Added this for debugging\n    fmt.Printf(\"%#v\\n\", certpool)\n\n    // We don't need to load the Client Cert\n    // cert, err := tls.LoadX509KeyPair(cfg.MQTT.ClientCert, cfg.MQTT.ClientKey)\n    // if err != nil {\n    // \treturn nil, fmt.Errorf(\"failed to load client certificate: %w\", err)\n    // }\n    // cert.Leaf, err = x509.ParseCertificate(cert.Certificate[0])\n    // if err != nil {\n    // \treturn nil, fmt.Errorf(\"failed to parse client certificate: %w\", err)\n    // }\n\n    return \u0026tls.Config{\n        RootCAs:            certpool,\n        InsecureSkipVerify: false,\n\n        // We don't need to load the Client Cert\n        // Certificates:       []tls.Certificate{cert},\n    }, nil\n}\n```\n\nEdit [`ttn-mqtt.yaml`](ttn-mqtt.yaml) and configure MQTT as follows...\n\n```yaml\n## Change au1.cloud.thethings.network to our MQTT Public Address\n## Use ssl://... instead of tcp://...\n## Port number is 8883 instead of 1883\nserver:      ssl://au1.cloud.thethings.network:8883\n\n## For TLS CA Certificate\nca_cert:     ttn.cer\nclient_cert: certs/xxxxx-certificate.pem.crt\nclient_key:  certs/xxxxx-private.pem.key\n```\n\n`client_cert` and `client_key` won't be used, but they must be uncommented.\n\nTo get `ttn.cer`: Browse to your Region-Specific URL for The Things Network, like...\n\n```text\nhttps://au1.cloud.thethings.network\n```\n\nIn the URL Bar, click...\n\nLock Icon → Connection Is Secure → Certificate Is Valid → Details → Copy To File → Base64 CER\n\nEnter `ttn.cer` as the filename.\n\nRestart `mqtt2prometheus`.\n\nWe should see...\n\n```text\n\u0026x509.CertPool{byName:map[string][]int{\"0\u00261$0\\\"\\x06\\x03U\\x04\\x03\\x13\\x1bau1.cloud.thethings.network\":[]int{0}}, lazyCerts:[]x509.lazyCert{x509.lazyCert{rawSubject:[]uint8{0x30, 0x26, 0x31, 0x24, 0x30, 0x22, 0x6, 0x3, 0x55, 0x4, 0x3, 0x13, 0x1b, 0x61, 0x75, 0x31, 0x2e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2e, 0x74, 0x68, 0x65, 0x74, 0x68, 0x69, 0x6e, 0x67, 0x73, 0x2e, 0x6e, 0x65, 0x74, 0x77, 0x6f, 0x72, 0x6b}, getCert:(func() (*x509.Certificate, error))(0x109da20)}}, haveSum:map[x509.sum224]bool{x509.sum224{0x4, 0xc3, 0x5, 0xf6, 0x3f, 0x5b, 0x23, 0xe3, 0xfb, 0xf9, 0x78, 0xd2, 0x49, 0xb4, 0xda, 0xa1, 0x75, 0x13, 0xe4, 0x2b, 0xf4, 0x7c, 0x97, 0x71, 0xf3, 0x67, 0x0, 0xa9}:true}}\n\nmqttclient/mqttClient.go:20      \nConnected to MQTT Broker\n\nweb/tls_config.go:191            \n{\"level\": \"info\", \"msg\": \"TLS is disabled.\", \"http2\": false}\n\nmqttclient/mqttClient.go:21      \nWill subscribe to topic {\"topic\": \"#\"}\n```\n\nIn case of problems, use WireShark to troubleshoot the TLS connection...\n\n![Wireshark sniffing MQTT TLS](https://lupyuen.github.io/images/prometheus-wireshark.png)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flupyuen%2Fprometheus-the-things-network","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flupyuen%2Fprometheus-the-things-network","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flupyuen%2Fprometheus-the-things-network/lists"}