{"id":21915602,"url":"https://github.com/clustercockpit/cc-metric-collector","last_synced_at":"2026-03-13T19:02:47.520Z","repository":{"id":38240471,"uuid":"339441254","full_name":"ClusterCockpit/cc-metric-collector","owner":"ClusterCockpit","description":"A node agent for measuring, processing and forwarding node level metrics","archived":false,"fork":false,"pushed_at":"2024-11-22T20:04:44.000Z","size":2465,"stargazers_count":4,"open_issues_count":10,"forks_count":8,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-11-22T21:19:47.451Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ClusterCockpit.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-02-16T15:24:10.000Z","updated_at":"2024-11-20T15:50:17.000Z","dependencies_parsed_at":"2023-01-30T18:16:37.922Z","dependency_job_id":"580cf335-75fc-4b33-b25d-4663dfa1bd84","html_url":"https://github.com/ClusterCockpit/cc-metric-collector","commit_stats":null,"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ClusterCockpit%2Fcc-metric-collector","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ClusterCockpit%2Fcc-metric-collector/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ClusterCockpit%2Fcc-metric-collector/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ClusterCockpit%2Fcc-metric-collector/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ClusterCockpit","download_url":"https://codeload.github.com/ClusterCockpit/cc-metric-collector/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":226971327,"owners_count":17711413,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-28T19:13:03.651Z","updated_at":"2026-03-13T19:02:47.507Z","avatar_url":"https://github.com/ClusterCockpit.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003c!--\n---\ntitle: cc-metric-collector\ndescription: Metric collecting node agent\ncategories: [cc-metric-collector]\ntags: ['Admin']\nweight: 2\nhugo_path: docs/reference/cc-metric-collector/_index.md\n---\n--\u003e\n\n# cc-metric-collector\n\nA node agent for measuring, processing and forwarding node level metrics. It is part of the [ClusterCockpit ecosystem](https://clustercockpit.org/docs/overview/).\n\nThe metric collector sends (and receives) metric in the [InfluxDB line protocol](https://docs.influxdata.com/influxdb/cloud/reference/syntax/line-protocol/) as it provides flexibility while providing a separation between tags (like index columns in relational databases) and fields (like data columns).\n\nThere is a single timer loop that triggers all collectors serially, collects the collectors' data and sends the metrics to the sink. This is done as all data is submitted with a single time stamp. The sinks currently use mostly blocking APIs.\n\nThe receiver runs as a go routine side-by-side with the timer loop and asynchronously forwards received metrics to the sink.\n\n\n[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.7438287.svg)](https://doi.org/10.5281/zenodo.7438287)\n\n\n# Configuration\n\nConfiguration is implemented using a single json document that is distributed over network and may be persisted as file.\nSupported metrics are documented [here](https://github.com/ClusterCockpit/cc-specifications/blob/master/interfaces/lineprotocol/README.md).\n\nThere is a main configuration file with basic settings that point to the other configuration files for the different components.\n\n``` json\n{\n  \"sinks-file\": \"sinks.json\",\n  \"collectors-file\" : \"collectors.json\",\n  \"receivers-file\" : \"receivers.json\",\n  \"router-file\" : \"router.json\",\n  \"main\": {\n    \"interval\": \"10s\",\n    \"duration\": \"1s\"\n  }\n}\n```\n\nThe `interval` defines how often the metrics should be read and send to the sink. The `duration` tells collectors how long one measurement has to take. This is important for some collectors, like the `likwid` collector. For more information, see [here](./docs/configuration.md).\n\nSee the component READMEs for their configuration:\n\n* [`collectors`](./collectors/README.md)\n* [`sinks`](https://github.com/ClusterCockpit/cc-lib/blob/main/sinks/README.md)\n* [`receivers`](https://github.com/ClusterCockpit/cc-lib/blob/main/receivers/README.md)\n* [`router`](./internal/metricRouter/README.md)\n\n# Installation\n\n```\n$ git clone git@github.com:ClusterCockpit/cc-metric-collector.git\n$ make (downloads LIKWID, builds it as static library with 'direct' accessmode and copies all required files for the collector)\n$ go get (requires at least golang 1.16)\n$ make\n```\n\nFor more information, see [here](./docs/building.md).\n\n# Running\n\n```\n$ ./cc-metric-collector --help\nUsage of metric-collector:\n  -config string\n    \tPath to configuration file (default \"./config.json\")\n  -log string\n    \tPath for logfile (default \"stderr\")\n  -once\n    \tRun all collectors only once\n```\n\n# Scenarios\n\nThe metric collector was designed with flexibility in mind, so it can be used in many scenarios. Here are a few:\n\n```mermaid\nflowchart TD\n  subgraph a [\"Cluster A\"]\n  nodeA[NodeA with CC collector]\n  nodeB[NodeB with CC collector]\n  nodeC[NodeC with CC collector]\n  end\n  a --\u003e db[(Database)]\n  db \u003c--\u003e ccweb(\"Webfrontend\")\n```\n\n``` mermaid\nflowchart TD\n  subgraph a [ClusterA]\n  direction LR\n  nodeA[NodeA with CC collector]\n  nodeB[NodeB with CC collector]\n  nodeC[NodeC with CC collector]\n  end\n  subgraph b [ClusterB]\n  direction LR\n  nodeD[NodeD with CC collector]\n  nodeE[NodeE with CC collector]\n  nodeF[NodeF with CC collector]\n  end\n  a --\u003e ccrecv{\"CC collector as receiver\"}\n  b --\u003e ccrecv\n  ccrecv --\u003e db[(\"Database1\")]\n  ccrecv -.-\u003e db2[(\"Database2\")]\n  db \u003c-.-\u003e ccweb(\"Webfrontend\")\n```\n\n# Contributing\n\nThe ClusterCockpit ecosystem is designed to be used by different HPC computing centers. Since configurations and setups differ between the centers, the centers likely have to put some work into the cc-metric-collector to gather all desired metrics.\n\nYou are free to open an issue to request a collector but we would also be happy about PRs.\n\n# Contact\n\n* [Matrix.org ClusterCockpit General chat](https://matrix.to/#/#clustercockpit-dev:matrix.org)\n* [Matrix.org ClusterCockpit Development chat](https://matrix.to/#/#clustercockpit:matrix.org)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclustercockpit%2Fcc-metric-collector","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fclustercockpit%2Fcc-metric-collector","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclustercockpit%2Fcc-metric-collector/lists"}