{"id":28455696,"url":"https://github.com/questdb/opcua-questdb-telegraf","last_synced_at":"2025-10-03T23:19:16.422Z","repository":{"id":241819206,"uuid":"807716167","full_name":"questdb/opcua-questdb-telegraf","owner":"questdb","description":"Playground for OPCUA data ingestion into QuestDB using telegraf","archived":false,"fork":false,"pushed_at":"2024-06-18T14:51:57.000Z","size":15,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-06-06T22:11:16.962Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/questdb.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-29T16:30:04.000Z","updated_at":"2024-06-18T14:52:00.000Z","dependencies_parsed_at":"2024-06-18T18:11:28.303Z","dependency_job_id":null,"html_url":"https://github.com/questdb/opcua-questdb-telegraf","commit_stats":null,"previous_names":["javier/opcua-questdb-telegraf","questdb/opcua-questdb-telegraf"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/questdb/opcua-questdb-telegraf","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/questdb%2Fopcua-questdb-telegraf","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/questdb%2Fopcua-questdb-telegraf/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/questdb%2Fopcua-questdb-telegraf/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/questdb%2Fopcua-questdb-telegraf/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/questdb","download_url":"https://codeload.github.com/questdb/opcua-questdb-telegraf/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/questdb%2Fopcua-questdb-telegraf/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278240526,"owners_count":25954226,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-03T02:00:06.070Z","response_time":53,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-06T22:10:26.112Z","updated_at":"2025-10-03T23:19:16.416Z","avatar_url":"https://github.com/questdb.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# OPC UA QuestDB Telegraf Integration\n\n\nThis project provides a setup to read data from an OPC UA server, process it using Telegraf, and store it in QuestDB.\nThe default configuration uses the public Milo OPC UA demo server at `opc.tcp://milo.digitalpetri.com:62541/milo`, but\nyou can also run a local instance of the Milo server following the instructions in the\n[Milo OPC UA Demo Server](https://github.com/digitalpetri/opc-ua-demo-server) repository.\n\n\n\n## Components\n\nThis repository has two main parts:\n\n* Telegraf configuration file, at `telegraf_opcua_milo_server.conf`, which sets up input from the OPC UA server, collects\nand transform the source data, and output to QuestDB.\n* Auxiliary Python scripts: In case you want to interactively explore the metrics available at the OPC UA Server to\nextend the existing telegraf config.\n\n\n## Getting Started\n\n### Prerequisites\n\n- Python 3.8+\n- Telegraf\n- QuestDB\n\n### Setup\n\n1. **Clone the repository**:\n   ```sh\n   git clone https://github.com/javier/opcua-questdb-telegraf.git\n   cd opcua-questdb-telegraf\n   ```\n\n2. **Set up environment variables**:\n\nThe behaviour of this integration can be configured via environment variables that are referenced at the telegraf\nconfiguration file. To make things easier, this repository provides a `telegraf_env` with sensible default values.\n\n```\nexport OPCUA_ENDPOINT=\"opc.tcp://milo.digitalpetri.com:62541/milo\"\n# Uncomment to use your own local Milo server\n# export OPCUA_ENDPOINT=\"opc.tcp://127.0.0.1:62541/milo\"\nexport TELEGRAF_LOG_FILE=\"/tmp/telegraf.log\"\nexport RANDOM_METRICS_TABLE_NAME=\"opcua_metrics\"\nexport SERVER_METRICS_TABLE_NAME=\"opcua_server_metrics\"\nexport QUESTDB_HTTP_ENDPOINT=\"http://127.0.0.1:9000\"\nexport QUESTDB_HTTP_TOKEN=\"\"\nexport QUESTDB_SKIP_TLS_VERIFICATION=true\n```\n\nIf you want to apply those defaults, or change anything you need and then apply the changes, just execute\n\n   ```sh\n   . ./telegraf_env\n   ```\n\n3. ** (optional) Install Python dependencies (only if using the auxiliary python scripts)**:\n\n   ```sh\n   pip install -r requirements.txt\n   ```\n\n### Start Collecting Data with Telegraf and QuestDB\n\nIf no environment variables were set, the defaults will be used. By default, the telegraf logs will be just to the standard\noutput. Please make sure QuestDB is up and running before starting Telegraf.\n\n   ```sh\n   telegraf --config telegraf_opcua_milo_server.conf\n   ```\n\nTelegraf will start and will connect to the server. You should see in the logs that metrics are collected every few seconds\nand are flushed a few times a minute. Once you start seeing metrics being flushed, you can connect to the QuestDB web console,\nwhich defaults to http://localhost:9000, and issue queries like:\n\n```sql\nSELECT * from opcua_metrics;\n\nSELECT * from opcua_server_metrics;\n```\n\n### The metrics being collected\n\nThe Milo demo server exposes several metrics. I choose some metrics from a namespace registered as \"Dynamic\" on the Milo OPC UA server, which just generate random integer and float numbers, and a few metrics from a namespace called \"ServerStatus\".\n\nSince metrics come from the server in a sparse formar (each value comes as a different item), I am using some telegraf processors to pre-process the metrics, and then a telegraf aggregator to produce dense rows, with several metrics for the\nsame timestamp in a single row. Please refer to the comments in the `telegraf_opcua_milo_server.conf ` to learn more about this.\n\nIf you want to explore which other metrics are available in the server, or you want to connect to a different OPC UA server, you will need an OPC UA explorer to find the relevant metrics, or you can use the auxiliary Python scripts to explore the entities available in your server.\n\n\n### Python Scripts\n\n- **`discover_nodes.py`**: This script browses the OPC UA server and lists the available nodes. It can output the results to a JSON file. By default it will connect to the online Milo server and will output only to standard output, but it can be controlledlike this\n```sh\npython discover_nodes.py --server-url opc.tcp://127.0.0.1:62541/milo --output-file remote-nodes.json\n```\n\n- **`dynamic_values_opcua.py`**: This script subscribes to the same nodes we are using in the telegraf config file and outputs its values to standard output.\n\n```sh\npython dynamic_values_opcua.py --server-url opc.tcp://127.0.0.1:62541/milo\n```\n\n- **`nodes_info.py`**: This script provides node IDs for the nodes in the UPC UA server, to identify which are the node names we would need to use in the telegraf\nconfig file if we want to collect extra metrics. It outputs to standard output, but since output can be large, it is advisable to redirect to a file.\n\n```sh\npython nodes_info.py \u003e out.txt\n```\n\n\n\n\n\n\n\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fquestdb%2Fopcua-questdb-telegraf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fquestdb%2Fopcua-questdb-telegraf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fquestdb%2Fopcua-questdb-telegraf/lists"}