{"id":22499738,"url":"https://github.com/opiproject/otel","last_synced_at":"2025-08-03T06:32:20.722Z","repository":{"id":104872752,"uuid":"594067224","full_name":"opiproject/otel","owner":"opiproject","description":"OPI Monitoring and Telemetry via Open Telemetry (OTEL)","archived":false,"fork":false,"pushed_at":"2024-10-29T23:43:23.000Z","size":186,"stargazers_count":9,"open_issues_count":7,"forks_count":14,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-10-30T01:53:42.500Z","etag":null,"topics":["dpu","logs","metrics","monitoring","ngnix","opentelemetry","opi","otel","redfish","spdk","telemetry"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/opiproject.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-01-27T14:22:32.000Z","updated_at":"2024-10-29T23:43:27.000Z","dependencies_parsed_at":"2024-07-10T22:47:21.031Z","dependency_job_id":"5c4760d7-2468-448f-b3a0-27ae39bea149","html_url":"https://github.com/opiproject/otel","commit_stats":{"total_commits":171,"total_committers":6,"mean_commits":28.5,"dds":"0.29824561403508776","last_synced_commit":"f4dae5e68ac5a14dac60964e2bf57d90fd618f02"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opiproject%2Fotel","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opiproject%2Fotel/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opiproject%2Fotel/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opiproject%2Fotel/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/opiproject","download_url":"https://codeload.github.com/opiproject/otel/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":228532151,"owners_count":17933247,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dpu","logs","metrics","monitoring","ngnix","opentelemetry","opi","otel","redfish","spdk","telemetry"],"created_at":"2024-12-06T22:15:43.486Z","updated_at":"2024-12-06T22:15:44.239Z","avatar_url":"https://github.com/opiproject.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Monitoring \u0026 Telemetry\n\n[![Docker](https://github.com/opiproject/otel/actions/workflows/docker-publish.yml/badge.svg)](https://github.com/opiproject/otel/actions/workflows/docker-publish.yml)\n[![Linters](https://github.com/opiproject/otel/actions/workflows/linters.yml/badge.svg)](https://github.com/opiproject/otel/actions/workflows/linters.yml)\n[![License](https://img.shields.io/github/license/opiproject/otel?style=flat-square\u0026color=blue\u0026label=License)](https://github.com/opiproject/otel/blob/master/LICENSE)\n[![GitHub stars](https://img.shields.io/github/stars/opiproject/otel.svg?style=flat-square\u0026label=github%20stars)](https://github.com/opiproject/otel)\n[![GitHub Contributors](https://img.shields.io/github/contributors/opiproject/otel.svg?style=flat-square)](https://github.com/opiproject/otel/graphs/contributors)\n\n## I Want To Contribute\n\nThis project welcomes contributions and suggestions.  We are happy to have the Community involved via submission of **Issues and Pull Requests** (with substantive content or even just fixes). We are hoping for the documents, test framework, etc. to become a community process with active engagement.  PRs can be reviewed by any number of people, and a maintainer may accept.\n\nSee [CONTRIBUTING](https://github.com/opiproject/opi/blob/main/CONTRIBUTING.md) and [GitHub basic process](https://github.com/opiproject/opi/blob/main/Policies/doc-github-rules.md) for more details.\n\n## Getting started\n\n:exclamation: `docker-compose` is deprecated. For details, see [Migrate to Compose V2](https://docs.docker.com/compose/migrate/).\n\n1. Run `docker compose up -d` or `docker-compose up -d`\n2. Open `http://localhost:3000/explore` for querying InfluxDB\n3. Open `http://localhost:9091/api/v1/query?query=dpu_bytes_read` for querying Prometheus\n\n## Intro\n\n- OPI adoped \u003chttps://opentelemetry.io/\u003e for DPUs\n- OPI goal is to pick 1 standard protocol that\n  - all vendors can implement (both linux and non-linux based)\n  - DPU consumers can integrate once in their existing monitoring systems and tools\n\n- OpenTelemetry suports those data sources\n  - Traces\n  - Metrics\n  - Logs\n\n## What is mandated by OPI\n\n- OpenTelemetry is made up of several main components:\n  - [Specification](https://github.com/open-telemetry/opentelemetry-specification)\n  - [Collector](https://github.com/open-telemetry/opentelemetry-collector) and [Collector Contrib](https://github.com/open-telemetry/opentelemetry-collector-contrib)\n  - SDKs (different programming languages), for example [Python](https://github.com/open-telemetry/opentelemetry-python), [Java](https://github.com/open-telemetry/opentelemetry-java) and [Go](https://github.com/open-telemetry/opentelemetry-go)\n\n- OPI is only mandating OTEL `Specification`\n- SDK and Collector specific implementation are left to the users\n  - They can be also from OTEL or other sources.\n\n## Collector deploy options\n\nSee \u003chttps://opentelemetry.io/docs/collector/deployment\u003e\n\n![OPI Telemetry Deploy Option](doc/dpu-otel.png)\n\n- OpenTelemetry collector supports several deployments:\n  - Deploy as side car inside every pod\n  - Deploy another one as aggregator per Node\n  - Deploy another one as super-aggregator per Cluster\n\n- The benefits of having multiple collectors at different levels are:\n  - Increased redundancy\n  - Enrichment\n  - Filtering\n  - Separating trust domains\n  - Batching\n  - Sanitization\n\n- Recommendation (reference)\n  - micro-aggregator inside each DPU/IPU\n  - macro-aggregator between DPUs\n    - macro-aggregator can run on the host with DPU/IPU or on a separate host\n\n## System Monitoring\n\n- System monitoring (cpu,mem,nic,...)\n  - see as example \u003chttps://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/hostmetricsreceiver\u003e\n\n- BMC monitoring (temp, power)\n  - Redfish extention for OTEL collector can be used to collect HW/BMC related metrics like temperature, power and others...\n  - For testing using mockup : `docker run --rm dmtf/redfish-mockup-server:1.1.8`\n\n- TBD\n  - OPI wants to define which telemetries are mandatory for each vendor to implement and which are optional\n\n## Storage SPDK\n\nsee \u003chttps://spdk.io/doc/jsonrpc_proxy.html\u003e\n\nUse this patch to handle chunked data \u003chttps://review.spdk.io/gerrit/c/spdk/spdk/+/12082\u003e\n\n```text\nsudo ./spdk/scripts/rpc_http_proxy.py 127.0.0.1 9009 spdkuser spdkpass\n```\n\nTest Proxy is running correctly\n\n```text\ncurl -k --user spdkuser:spdkpass -X POST -H \"Content-Type: application/json\" -d '{\"id\": 1, \"method\": \"bdev_get_bdevs\", \"params\": {\"name\": \"Malloc0\"}}' http://127.0.0.1:9009/\n```\n\n## Tracing\n\n- Tracing inside DPU/IPU (more tight SDK integration into our service and IPDK), streaming to zipkin/jaeger\n- TODO: need more details and examples\n\n## Logging\n\n- For example \u003chttps://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/syslogreceiver\u003e\n- TODO: need more details and examples\n\n## Query examples\n\n```text\ncurl --fail http://127.0.0.1:9091/api/v1/query?query=mem_free | grep mem_free\ncurl --fail http://127.0.0.1:9091/api/v1/query?query=cpu_usage_user | grep cpu_usage_user\ncurl --fail http://127.0.0.1:9091/api/v1/query?query=dpu_num_blocks | grep dpu_num_blocks\ncurl --fail http://127.0.0.1:9091/api/v1/query?query=net_bytes_recv_total | grep net_bytes_recv_total\ncurl --fail http://127.0.0.1:9091/api/v1/query?query=nginx_requests | grep nginx_requests\ncurl --fail http://127.0.0.1:9091/api/v1/query?query=redfish_thermal_fans_reading_rpm | grep redfish_thermal_fans_reading_rpm\ncurl --fail http://127.0.0.1:9091/api/v1/query?query=disk_used_percent | grep disk_used_percent\ncurl --fail http://127.0.0.1:9091/api/v1/query?query=netstat_tcp_listen | grep netstat_tcp_listen\ncurl --fail http://127.0.0.1:9091/api/v1/query?query=dns_query_result_code | grep dns_query_result_code\ncurl --fail http://127.0.0.1:9091/api/v1/query?query=ethtool_duplex | grep ethtool_duplex\ncurl --fail http://127.0.0.1:9091/api/v1/query?query=kernel_boot_time_total | grep kernel_boot_time_total\n```\n\n## Running example\n\n- Make sure you run `rpc_http_proxy`, `spdk` and `ngnix` apps from above.\n- Consider adjusting SPDK IP address in [telegraf.conf](https://github.com/opiproject/otel/blob/1e00ed289762f36fea65e16ace64611a3282be2d/config/telegraf.conf#L8) from `http://spdk:9009` to `http://localhost:9009`.\n- Consider adjusting NGNIX IP address in [telegraf.conf](https://github.com/opiproject/otel/blob/1e00ed289762f36fea65e16ace64611a3282be2d/config/telegraf.conf#L21) from `http://web:80/nginx_status` to `http://localhost:80/nginx_status`.\n- Consider adjusting BMC address and credentials in [telegraf.conf](https://github.com/opiproject/otel/blob/1e00ed289762f36fea65e16ace64611a3282be2d/config/telegraf.conf#L2).\n\n```text\n$ docker run --rm --net=host -v $(pwd)/telegraf.conf:/etc/telegraf/telegraf.conf:ro telegraf:1.22\n2022-03-29T18:47:11Z I! Using config file: /etc/telegraf/telegraf.conf\n2022-03-29T18:47:11Z I! Starting Telegraf 1.22.0\n2022-03-29T18:47:11Z I! Loaded inputs: http\n2022-03-29T18:47:11Z I! Loaded aggregators:\n2022-03-29T18:47:11Z I! Loaded processors:\n2022-03-29T18:47:11Z W! Outputs are not used in testing mode!\n2022-03-29T18:47:11Z I! Tags enabled: host=localhost\n\ndpu,host=866da842daa4,name=Malloc0,url=http://spdk:9009 write_latency_ticks=0,num_unmap_ops=0,num_read_ops=2,num_write_ops=0,unmap_latency_ticks=0,bytes_read=36864,bytes_unmapped=0,read_latency_ticks=323535,bytes_written=0 1674846640000000000\ndpu,host=866da842daa4,name=Malloc1,url=http://spdk:9009 bytes_read=36864,read_latency_ticks=92186,bytes_unmapped=0,num_unmap_ops=0,unmap_latency_ticks=0,write_latency_ticks=0,num_read_ops=2,bytes_written=0,num_write_ops=0 1674846640000000000\n\nmem,host=52ee5c75df01 commit_limit=69312983040i,committed_as=13494636544i,huge_page_size=2097152i,used_percent=10.100053796757296,high_free=0i,inactive=13541511168i,low_free=0i,shared=3904901120i,sreclaimable=812650496i,swap_cached=0i,free=100370612224i,huge_pages_total=2048i,low_total=0i,page_tables=49500160i,used=13567504384i,huge_pages_free=1357i,mapped=901996544i,slab=2243977216i,swap_total=4294963200i,cached=20385955840i,vmalloc_chunk=0i,vmalloc_used=0i,write_back=0i,swap_free=4294963200i,high_total=0i,available_percent=86.20598148102354,available=115801366528i,sunreclaim=1431326720i,total=134331011072i,buffered=6938624i,dirty=856064i,vmalloc_total=14073748835531776i,write_back_tmp=0i,active=8859537408i 1650954170000000000\n\nnet,host=52ee5c75df01,interface=eth0 drop_in=0i,drop_out=0i,bytes_sent=16589i,bytes_recv=13986i,packets_sent=89i,packets_recv=110i,err_in=0i,err_out=0i 1650954170000000000\n\nnginx,host=6465dac4e3a5,port=80,server=web waiting=0i,active=1i,accepts=1i,handled=1i,requests=2i,reading=0i,writing=1i 1677179010000000000\n\ncpu,cpu=cpu0,host=52ee5c75df01 usage_user=99.6999999973923,usage_system=0.09999999999763531,usage_idle=0,usage_iowait=0,usage_softirq=0,usage_steal=0,usage_nice=0,usage_irq=0.19999999999527063,usage_guest=0,usage_guest_nice=0 1650954170000000000\ncpu,cpu=cpu1,host=52ee5c75df01 usage_user=99.70000000204891,usage_system=0,usage_irq=0.2999999999974534,usage_steal=0,usage_idle=0,usage_nice=0,usage_iowait=0,usage_softirq=0,usage_guest=0,usage_guest_nice=0 1650954170000000000\ncpu,cpu=cpu2,host=52ee5c75df01 usage_system=0,usage_idle=0,usage_iowait=0,usage_guest_nice=0,usage_user=99.79999999981374,usage_nice=0,usage_irq=0.20000000000436557,usage_softirq=0,usage_steal=0,usage_guest=0 1650954170000000000\ncpu,cpu=cpu3,host=52ee5c75df01 usage_guest_nice=0,usage_user=99.79999999981374,usage_idle=0,usage_nice=0,usage_iowait=0,usage_guest=0,usage_system=0,usage_irq=0.20000000000436557,usage_softirq=0,usage_steal=0 1650954170000000000\ncpu,cpu=cpu4,host=52ee5c75df01 usage_user=99.70029970233988,usage_guest=0,usage_steal=0,usage_guest_nice=0,usage_system=0.09990009990223975,usage_idle=0,usage_nice=0,usage_iowait=0,usage_irq=0.19980019979993657,usage_softirq=0 1650954170000000000\ncpu,cpu=cpu5,host=52ee5c75df01 usage_nice=0,usage_iowait=0,usage_irq=0.2997002997044478,usage_softirq=0,usage_steal=0,usage_guest_nice=0,usage_user=99.70029970233988,usage_idle=0,usage_guest=0,usage_system=0 1650954170000000000\n\nredfish_thermal_temperatures,address=bmc,health=OK,host=fd287855dfb3,member_id=0,name=CPU1\\ Temp,rack=WEB43,row=North,source=web483,state=Enabled reading_celsius=41,upper_threshold_critical=45,upper_threshold_fatal=48 1659628400000000000\nredfish_thermal_temperatures,address=bmc,host=fd287855dfb3,member_id=1,name=CPU2\\ Temp,rack=WEB43,row=North,source=web483,state=Disabled upper_threshold_critical=45,upper_threshold_fatal=48 1659628400000000000\nredfish_thermal_temperatures,address=bmc,health=OK,host=fd287855dfb3,member_id=2,name=Chassis\\ Intake\\ Temp,rack=WEB43,row=North,source=web483,state=Enabled reading_celsius=25,upper_threshold_critical=40,upper_threshold_fatal=50,lower_threshold_critical=5,lower_threshold_fatal=0 1659628400000000000\n\nredfish_thermal_fans,address=bmc,health=OK,host=fd287855dfb3,member_id=0,name=BaseBoard\\ System\\ Fan,rack=WEB43,row=North,source=web483,state=Enabled lower_threshold_fatal=0i,reading_rpm=2100i 1659628400000000000\nredfish_thermal_fans,address=bmc,health=OK,host=fd287855dfb3,member_id=1,name=BaseBoard\\ System\\ Fan\\ Backup,rack=WEB43,row=North,source=web483,state=Enabled lower_threshold_fatal=0i,reading_rpm=2050i 1659628400000000000\n\nredfish_power_powersupplies,address=bmc,health=Warning,host=fd287855dfb3,member_id=0,name=Power\\ Supply\\ Bay,rack=WEB43,row=North,source=web483,state=Enabled line_input_voltage=120,last_power_output_watts=325,power_capacity_watts=800 1659628400000000000\n\nredfish_power_voltages,address=bmc,health=OK,host=fd287855dfb3,member_id=0,name=VRM1\\ Voltage,rack=WEB43,row=North,source=web483,state=Enabled lower_threshold_critical=11,lower_threshold_fatal=10,reading_volts=12,upper_threshold_critical=13,upper_threshold_fatal=15 1659628400000000000\nredfish_power_voltages,address=bmc,health=OK,host=fd287855dfb3,member_id=1,name=VRM2\\ Voltage,rack=WEB43,row=North,source=web483,state=Enabled reading_volts=5,upper_threshold_critical=7,lower_threshold_critical=4.5 1659628400000000000\n\n...\n```\n\n## questions to  (eventually remove this section)\n\n- Is there integration of OTEL with kvm or esx ?\n- Use case of standalone DPU, not attached to server. Still runs OTEL collector\n\n## Working items\n\n- [#7](/../../issues/7) Starting new workstream to find out set of common metrics across vendors that OPI will mandate\n  - Action items on Marvell, Nvidia, Intell to come up with the list and present on the next meeting\n- [#6](/../../issues/6) Starting new POC with OTEL SDK and hello world app\n  - Action items on Nvidia to help compiling DOCA with OTEL SDK (i.e. \u003chttps://github.com/open-telemetry/opentelemetry-cpp\u003e ) and hello world app to show metrics/traces streaming to standard ecosystem (zipkin/grafana/elastic/...)\n- [#5](/../../issues/5) Continue working on existing telegraf example and enhance with more metrics\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopiproject%2Fotel","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopiproject%2Fotel","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopiproject%2Fotel/lists"}