{"id":23080088,"url":"https://github.com/opsdis/tempo_trace_aggregation","last_synced_at":"2025-08-15T22:30:43.066Z","repository":{"id":42024407,"uuid":"473605423","full_name":"opsdis/tempo_trace_aggregation","owner":"opsdis","description":"Tempo trace aggregation, tta, is an example how to create a dynamic service map in Grafana with the nodegraph plugin based on traces stored in Tempo","archived":false,"fork":false,"pushed_at":"2022-11-16T06:44:32.000Z","size":323,"stargazers_count":2,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"master","last_synced_at":"2023-03-06T06:32:13.159Z","etag":null,"topics":["grafana","tempo"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/opsdis.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-03-24T12:54:15.000Z","updated_at":"2022-11-18T12:36:25.000Z","dependencies_parsed_at":"2022-08-12T02:40:35.745Z","dependency_job_id":null,"html_url":"https://github.com/opsdis/tempo_trace_aggregation","commit_stats":null,"previous_names":[],"tags_count":null,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opsdis%2Ftempo_trace_aggregation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opsdis%2Ftempo_trace_aggregation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opsdis%2Ftempo_trace_aggregation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opsdis%2Ftempo_trace_aggregation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/opsdis","download_url":"https://codeload.github.com/opsdis/tempo_trace_aggregation/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":229964404,"owners_count":18152034,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["grafana","tempo"],"created_at":"2024-12-16T13:04:29.348Z","updated_at":"2024-12-16T13:04:30.082Z","avatar_url":"https://github.com/opsdis.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"![Docker Pulls](https://img.shields.io/docker/pulls/athenodon/tempo_trace_aggregation)\n\nTempo trace aggregation - tta\n-----------------------------\n\n\u003e The main purpose of this project is to be a demo for [nodegraph-provider](https://github.com/opsdis/nodegraph-provider)\n\n\n# Overview\nTempo trace aggregation, tta, is an **example** how to create a dynamic service\nmap in Grafana with the [nodegraph plugin](https://grafana.com/docs/grafana/latest/visualizations/node-graph/) \nbased on traces stored in [Grafana Tempo](https://github.com/grafana/tempo). \n\n![Overview](docs/overview.png?raw=true \"Overview\")\n\nIt requires the project Nodegraph-provider, https://github.com/opsdis/nodegraph-provider.\nPlease read the [README](https://github.com/opsdis/nodegraph-provider/blob/master/README.md) \nhow the setup is done in Grafana.\n\n![Petclinic Graph](docs/petclinic.png?raw=true \"Example graph\")\n\n\n# Query Tempo\nTo get traces out of tempo we need to select on a tag name, default is `service.name`.\nThere is also possible just to filter on specific of the tag values of tag name\nwith regex. Default regex is `.*`\nBy default tta will create an additional node for the selected tag, this is often a benfit to get \na graph that is fully connected.\n\n# Requirements\n- A tempo installation or just create a free account on [Grafana Cloud](https://grafana.com/products/cloud/)\n- An installation of the [nodegraph-provider](https://github.com/opsdis/nodegraph-provider)\n- Python 3.8 on the server you will be running tta. \n\n# Get started\n\nInstall python dependency\n\n    pip install -r requierments.txt\n\nSee all tta options\n```\npython -m tempo_trace_aggregation -h \n\nusage: __main__.py [-h] [-g GRAPH] [-t TAG] [-f TAG_FILTER] [-n] [-T SERVICE_NODE_SUB_TITLE] [-L TRACE_THRESHOLD_MS] [-l LOOP_INTERVAL] [-c CONFIG] [-s SEARCH_FROM] [-m SEARCH_MODE]\n\ntta - Tempo trace aggregation\n\noptional arguments:\n  -h, --help            show this help message and exit\n  -g GRAPH, --graph GRAPH\n                        graph model in nodegraph-provider, no default value\n  -t TAG, --tag TAG     tag name to query on, default service.name\n  -f TAG_FILTER, --filter TAG_FILTER\n                        tag filter for the the tag value, default .*\n  -n, --not_use_tag_as_node\n                        use tag as a node, default true\n  -T SERVICE_NODE_SUB_TITLE, --service_node_sub_title SERVICE_NODE_SUB_TITLE\n                        the subTitle name, if use tag as a node, default 'Service Node'\n  -L TRACE_THRESHOLD_MS, --trace_threshold_ms TRACE_THRESHOLD_MS\n                        the trace threshold in ms that should indicate red on the graph node, default is 40.0\n  -l LOOP_INTERVAL, --loop_interval LOOP_INTERVAL\n                        loop with interval defined, default 0 sec, which means no looping\n  -c CONFIG, --config CONFIG\n                        config file for connections, default config.yml\n  -s SEARCH_FROM, --search_from SEARCH_FROM\n                        the number of seconds to search back in time, default 7200 sec (2h)\n  -m SEARCH_MODE, --search_mode SEARCH_MODE\n                        the Tempo search mode, available values are blocks, ingesters or all, default ingester\n\n```\n\nExample\n\n     python -m tempo_trace_aggregation -t service.name -s 1200 -l 120 -m ingester\n\nThis will collect traces that have a label called `service.name` and aggregate on the \nlabel value of `service.name`. Only traces that exist in the ingester and happened in the last 1200 second will be \naggregated and tta will do a new search every 120 second.\n\nPlease check out the command options and the example config file, `config_example.yml`, \nwhere all connection information for tempo and nodegraph-provider must exist.\n\n# Build docker\n\nUse the Dockerfile in the root directory of the project\n\n     docker build -t tempo_trace_aggregation .\n\nTo run the image a config file must be mounted\n\n     docker run -v $(pwd)/config.yaml:/app/config.yaml tempo_trace_aggregation\n\n## Docker compose example\n\n\u003e Build image for [nodegraph-provider](https://github.com/opsdis/nodegraph-provider) before running `docker-compose up`\n\nIn the directory `docker-compose` there is a complete example to set up a running example with nodegraph-provider, \nredis with graph module and tempo_trace_aggregation.\nThe example is based on selecting trace tags `service.name`. Just go into the file called `tta_config.yml` and update\nthe connection to tempo.\n\n# Nodegraph-provider configuration\nThe following nodegraph-provider configuration has a schema configuration that works with\ntta.\n\n```yml\n# Default values are commented\n# All default values can be overridden using environment values using NODEGRAPH_PROVIDER_XYZ\n# Nested values should be separated with _\n\n\n# API port\n# port: 9393\n# Configuration file name default without postfix\n#config: config\n# The prefix of the metrics\n#prefix: nodegraph_provider\n\n\nredis:\n#  host: \"localhost\"\n#  port: \"6379\"\n#  db: 0\n  #maxactive: 350\n#  max_idle: 10\n\n\n# The following do not have any default values\n\n# The graph_schema define the field name and data type for the output to the data source.\n# The field names are also used for the api calls to create, update and delete the nodes and edges.\n# The only field not used in these api calls are the edge id that is automatically set to sourceid:targetid of\n#  the nodes\ngraph_schemas:\n  micro:\n    # An example\n    node_fields:\n      - field_name: \"id\"\n        type: \"string\"\n      - field_name: \"title\"\n        type: \"string\"\n      - field_name: \"subTitle\"\n        type: \"string\"\n      - field_name: \"mainStat\"\n        type: \"number\"\n        displayName: \"Count\"\n      - field_name: \"secondaryStat\"\n        type: \"number\"\n        displayName: \"Avg duration (ms)\"\n      - field_name: \"arc__failed\"\n        type: \"number\"\n        color: \"red\"\n      - field_name: \"arc__passed\"\n        type: \"number\"\n        color: \"green\"\n      - field_name: \"detail__role\"\n        type: \"string\"\n        displayName: \"Role\"\n    edge_fields:\n      # id is never used in the api, will be set dynamically to nodes sourceid:targetid\n      - field_name: \"id\"\n        type: \"string\"\n      - field_name: \"source\"\n        type: \"string\"\n      - field_name: \"target\"\n        type: \"string\"\n      - field_name: \"mainStat\"\n        type: \"number\"\n      - field_name: \"secondaryStat\"\n        type: \"number\"\n      - field_name: \"detail__traffic\"\n        type: \"string\"\n        displayName: \"Traffic\"\n\n\n```\n# Metrics explained\nOn the node the mainStat is a counter on the number of times it has been \"active\" in the\ntraces. The secondaryStat is the average latency time of the traces.\nOn the edge the mainStat is always 1, and the secondaryStat is always 0. \n\nSo there is room for improvements for your specific use case. I have used the petclinic springboot microservice\nproject for testing, https://github.com/spring-petclinic/spring-petclinic-microservices.\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopsdis%2Ftempo_trace_aggregation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopsdis%2Ftempo_trace_aggregation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopsdis%2Ftempo_trace_aggregation/lists"}