{"id":19523898,"url":"https://github.com/eitansuez/istio-metrics-merge-example","last_synced_at":"2026-06-30T15:31:42.812Z","repository":{"id":96637522,"uuid":"530764350","full_name":"eitansuez/istio-metrics-merge-example","owner":"eitansuez","description":"An example of how Istio metrics merging works","archived":false,"fork":false,"pushed_at":"2022-08-30T22:35:57.000Z","size":67,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-02-26T01:27:03.598Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Kotlin","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/eitansuez.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-08-30T17:39:07.000Z","updated_at":"2022-08-30T17:39:25.000Z","dependencies_parsed_at":null,"dependency_job_id":"74a9f4af-64aa-45d9-b6e2-01764c718d5c","html_url":"https://github.com/eitansuez/istio-metrics-merge-example","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/eitansuez/istio-metrics-merge-example","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eitansuez%2Fistio-metrics-merge-example","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eitansuez%2Fistio-metrics-merge-example/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eitansuez%2Fistio-metrics-merge-example/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eitansuez%2Fistio-metrics-merge-example/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/eitansuez","download_url":"https://codeload.github.com/eitansuez/istio-metrics-merge-example/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eitansuez%2Fistio-metrics-merge-example/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34973589,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-30T02:00:05.919Z","response_time":92,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-11T00:45:07.392Z","updated_at":"2026-06-30T15:31:42.778Z","avatar_url":"https://github.com/eitansuez.png","language":"Kotlin","funding_links":[],"categories":[],"sub_categories":[],"readme":"2022.08.30\n\n# Hello boot / test metrics\n\nThis application is part of an exercise to comprehend how [metrics merging](https://istio.io/latest/docs/ops/integrations/prometheus/) works in Istio.\n\n## Description of metrics merging (as I understand it)\n\nThe istio-agent process in the sidecar exposes the scrape endpoint to prometheus.\n\nThis scrape endpoint is implemented by merging (or aggregating) the metrics from two sources:\n\n- the application's scrape endpoint\n- the envoy sidecar's scrape endpoint, which collects its own networking-related metrics.\n\n### Related\n\n[Writeup by Rob Salmond](https://superorbital.io/journal/istio-metrics-merging/), which is a little obfuscated, but worth a read.\n\nHis blog entry references the [metrics merging design doc](https://docs.google.com/document/d/1TTeN4MFmh4aUYYciR4oDBTtJsxl5-T5Tu3m3mGEdSo8/view).\n\nThe design doc mentions the construction of an \"internal\" environment variable ISTIO_PROMETHEUS_ANNOTATIONS derived from the information in the annotations, that tells the agent the url of the app/workload's scrape endpoint.\n\n## How to communicate the url of the app scrape endpoint\n\nThe convention appears to be as follows:\n\nOn the app workload pod manifest, add the annotations:\n\n```yaml\nannotations:\n  prometheus.io/scrape: \"true\"\n  prometheus.io/port: \"8080\"\n  prometheus.io/path: \"/actuator/prometheus/\"\n```\n\nThe istio-agent will use that information to locate the application's scrape endpoint, and take care to aggregate the metrics together with envoy's collected metrics/scrape endpoint.\n\n## Building the app\n\n1. Produce a local image named `test-metrics:\u003cversion\u003e` (the version is specified in the gradle build file), tag it and push it to a local registry (assumption: using a local k8s cluster configured with a local registry exposed to k8s on port 5000 and exposed on localhost on `\u003cport\u003e`, see [how to do this with k3d](https://k3d.io/v5.4.6/usage/registries/#create-a-dedicated-registry-together-with-your-cluster)).\n\n    ```\n    gradle bootBuildImage --imageName=localhost:\u003cport\u003e/test-metrics:\u003cversion\u003e --publishImage\n    ```\n\n## To test this\n\n1. Create a hello-world style spring boot app with the micrometer prometheus dependency and check that you can hit that endpoint (this project).\n\n2. Deploy the Istio Prometheus add-on.\n\n    ```shell\n    k apply -f ${ISTIO_DIR}/samples/addons/prometheus.yaml\n    ```\n\n3. Construct a k8s deployment manifest for this app (see manifests/hello-boot.yaml), and deploy the app to an istio-enabled k8s cluster.\n\n    ```shell\n    k apply -f manifests/hello-boot.yaml\n    ```\n\n4. Verify that the workload metrics endpoint indeed functions.\n\n    ```shell\n    k exec -it helloboot-v1-\u003ctab\u003e -c istio-proxy -- curl localhost:8080/actuator/prometheus\n    ```\n\n5. Check envoy's prometheus scrape endpoint.\n\n    ```shell\n    k exec -it helloboot-v1-\u003ctab\u003e -c istio-proxy -- curl localhost:15090/stats/prometheus\n    ```\n\n6. Locate and check the value of the ISTIO_PROMETHEUS_ANNOTATIONS environment variable inside the sidecar container.\n\n    ```shell\n    k exec -it helloboot-v1-\u003ctab\u003e -c istio-proxy -- /bin/sh\n    echo $ISTIO_PROMETHEUS_ANNOTATIONS\n    ```\n\n    The output should look like this:\n\n    ```json\n    {\"scrape\":\"true\",\"path\":\"/actuator/prometheus\",\"port\":\"8080\"}\n    ```\n\n8. Verify that the aggregated endpoint response includes both envoy and app metrics\n\n    ```shell\n    k exec -it helloboot-v1-585487d5f7-9d27j -c istio-proxy -- curl localhost:15020/stats/prometheus\n    ```\n\n9. Give it a couple of minutes, and check the prometheus dashboard for the presence of app workload metrics, e.g. `http_server_requests_seconds_count` or `jvm_memory_max_bytes` (or any custom metrics you choose to expose in your app via micrometer).\n\n    ```shell\n    istioctl dashboard prometheus\n    ```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feitansuez%2Fistio-metrics-merge-example","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feitansuez%2Fistio-metrics-merge-example","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feitansuez%2Fistio-metrics-merge-example/lists"}