{"id":19270361,"url":"https://github.com/netdata/agent-service-discovery","last_synced_at":"2025-04-21T20:33:40.102Z","repository":{"id":41863576,"uuid":"261711929","full_name":"netdata/agent-service-discovery","owner":"netdata","description":null,"archived":true,"fork":false,"pushed_at":"2025-01-30T10:07:12.000Z","size":320,"stargazers_count":15,"open_issues_count":2,"forks_count":10,"subscribers_count":11,"default_branch":"master","last_synced_at":"2025-02-27T03:05:03.551Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/netdata.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-05-06T09:29:03.000Z","updated_at":"2025-01-30T10:09:01.000Z","dependencies_parsed_at":"2023-02-16T10:31:22.334Z","dependency_job_id":"c3c91d74-5f24-493e-9e89-1b8f587f13c9","html_url":"https://github.com/netdata/agent-service-discovery","commit_stats":null,"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netdata%2Fagent-service-discovery","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netdata%2Fagent-service-discovery/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netdata%2Fagent-service-discovery/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netdata%2Fagent-service-discovery/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/netdata","download_url":"https://codeload.github.com/netdata/agent-service-discovery/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250128494,"owners_count":21379523,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T20:24:26.814Z","updated_at":"2025-04-21T20:33:40.089Z","avatar_url":"https://github.com/netdata.png","language":"Go","readme":"\u003c!--\ntitle: \"Service discovery\"\ncustom_edit_url: \"https://github.com/netdata/agent-service-discovery/edit/master/README.md\"\nsidebar_label: \"Service discovery\"\nlearn_status: \"Published\"\nlearn_rel_path: \"Integrations/Monitor/Anything\"\nsidebar_position: 300\n--\u003e\n\n# Service discovery\n\n\u003e [!WARNING]\n\u003e\n\u003e **Deprecation Notice**: This repository's service discovery functionality has been migrated to go.d.plugin in the main [Netdata repository](https://github.com/netdata/netdata). All future development, maintenance, and updates will continue there.\n\n\u003cdetails\u003e\n\u003csummary\u003eOld readme\u003c/summary\u003e\n\nService discovery extracts all the potentially useful information from different sources, converts it to the\nconfigurations and exports them to the different destinations.\n\n## Pipeline\n\nThe service discovery pipeline has four jobs:\n\n|           Job           | Description                                                                                                                                                                     |\n|:-----------------------:|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| [discovery](#Discovery) | Dynamically discovers monitoring targets by collecting events from kubernetes API server. It collects POD and SERVICE events.                                                   |\n|       [tag](#Tag)       | Dynamically add tags to discovered monitoring targets. Based on the POD and SERVICE fields and using patterns on them, one or more tags are attached to the monitoring targets. |\n|     [build](#Build)     | Dynamically creates data collection configurations for the monitored targets, using templates.                                                                                  |\n|    [export](#Export)    | Dynamically exports data collection configurations to allow netdata data collection plugins to use them. Data collection jobs in netdata are created and destroyed as needed.   |\n\nRouting in a job and between jobs based on `tags` and `selector`.\n\nPipeline configuration:\n\n```yaml\nname: \u003cname\u003e\ndiscovery: \u003cdiscovery_config\u003e\ntag: \u003ctag_config\u003e\nbuild: \u003cbuild_config\u003e\nexport: \u003cexport_config\u003e\n```\n\n## Tags and selectors\n\nTag, build and export jobs have `selector`, the pipeline routes a target/config to the job only if its tags matches job\nselectors.\n\nBoth tags and selector are just lists of words.\n\nA word must match the regex `^[a-zA-Z][a-zA-Z0-9=_.]*$`.\n\nTags special cases:\n\n- `-word`: the word will be removed on tags merging.\n\nSelectors special cases:\n\n- `!word`: shouldn’t contain the word.\n- `word|word|word`: should contain any word.\n\n## Discovery\n\nDiscovery job dynamically discovers targets using one of the supported service-discovery mechanisms.\n\nSupported mechanisms:\n\n- [kubernetes](#Kubernetes)\n\nDiscovery configuration:\n\n```yaml\nk8s:\n  - \u003ckubernetes_discovery_config\u003e\n```\n\n### Kubernetes\n\nKubernetes discoverer retrieves targets from [Kubernetes'](https://kubernetes.io/)\n[REST API](https://kubernetes.io/docs/reference/). It always stays synchronized with the cluster state.\n\nConfiguration options:\n\n```yaml\n# Mandatory. Tags to add to all discovered targets.\ntags: \u003ctags\u003e\n\n# Mandatory. The Kubernetes role of entities that should be discovered.\nrole: \u003crole\u003e\n\n# Optional. Discover only targets that exist on the same node as service-discovery.\n# This option works only for 'pod' role and it requires MY_NODE_NAME env variable to be set.\nlocal_mode: \u003cboolean\u003e\n\n# Optional. If omitted, all namespaces are used.\nnamespaces:\n  - \u003cnamespace\u003e\n```\n\nOne of the following role types can be configured to discover targets:\n\n- `pod`\n- `service`\n\n#### Pod Role\n\nThe pod role discovers all pods and exposes their containers as targets. For each declared port of a container, it\ngenerates single target. If there is no declared port it generates one target with empty `Port`, `PortName`\nand `PortProtocol` fields.\n\nAvailable pod target fields:\n\n| Name             | Type              | Value                                                     |\n|:-----------------|:------------------|:----------------------------------------------------------|\n| `TUID`           | string            | `Namespace_Name_ContName_PortProtocol_Port`               |\n| `Address`        | string            | `PodIP:Port`                                              |\n| `Namespace`      | string            | _pod.metadata.namespace_                                  |\n| `Name`           | string            | _pod.metadata.name_                                       |\n| `Annotations`    | map[string]string | _pod.metadata.annotations_                                |\n| `Labels`         | map[string]string | _pod.metadata.labels_                                     |\n| `NodeName`       | string            | _pod.spec.nodeName_                                       |\n| `PodIP`          | string            | _pod.status.podIP_                                        |\n| `ControllerName` | string            | _pod.OwnerReferences.Controller.Name_                     |\n| `ControllerKind` | string            | _pod.OwnerReferences.Controller.Kind_                     |\n| `ContName`       | string            | _pod.spec.containers.name_                                |\n| `Image`          | string            | _pod.spec.containers.image_                               |\n| `Env`            | map[string]string | _pod.spec.containers.env_ + _pod.spec.containers.envFrom_ |\n| `Port`           | string            | _pod.spec.containers.ports.containerPort_                 |\n| `PortName`       | string            | _pod.spec.containers.ports.name_                          |\n| `PortProtocol`   | string            | _pod.spec.containers.ports.protocol_                      |\n\n#### Service Role\n\nThe service role discovers a target for each service port for each service.\n\nAvailable service target fields:\n\n| Name           | Type              | Value                                     |\n|:---------------|:------------------|:------------------------------------------|\n| `TUID`         | string            | `Namespace_Name_PortProtocol_Port`        |\n| `Address`      | string            | `Name.Namespace.svc:Port`                 |\n| `Namespace`    | string            | _svc.metadata.namespace_                  |\n| `Name`         | string            | _svc.metadata.name_                       |\n| `Annotations`  | map[string]string | _svc.metadata.annotations_                |\n| `Labels`       | map[string]string | _svc.metadata.labels_                     |\n| `Port`         | string            | _pod.spec.containers.ports.containerPort_ |\n| `PortName`     | string            | _pod.spec.containers.ports.name_          |\n| `PortProtocol` | string            | _pod.spec.containers.ports.protocol_      |\n| `ClusterIP`    | string            | _svc.spec.clusterIP_                      |\n| `ExternalName` | string            | _svc.spec.externalName_                   |\n| `Type`         | string            | _svc.spec.ports.type_                     |\n\n## Tag\n\nTag job tags targets discovered by [discovery job](#Discovery). Its purpose is service identification.\n\nConfiguration is a list of tag rules:\n\n```yaml\n- \u003ctag_rule_config\u003e\n```\n\nTag rule configuration options:\n\n```yaml\n# Mandatory. Routes targets to this tag rule with tags matching this selector.\nselector: \u003cselector\u003e\n\n# Mandatory. Tags to merge with the target tags if at least on of the match rules matches.\ntags: \u003ctags\u003e\n\n# Mandatory. Match rules, at least one should be defined. \nmatch:\n  # Optional. Routes targets to this match rule with tags matching this selector.\n  - selector: \u003cselector\u003e\n\n    # Mandatory. Tags to merge with the target tags if this rule expression evaluates to true.\n    tags: \u003ctags\u003e\n\n    # Mandatory. Match expression.\n    expr: \u003cexpression\u003e\n```\n\n**Match expression evaluation result should be true or false**.\n\nExpression syntax is [go-template](https://golang.org/pkg/text/template/).\n\n### Available functions\n\n- go-template [built-in functions](https://golang.org/pkg/text/template/#hdr-Functions).\n- [sprig functions](http://masterminds.github.io/sprig/).\n- custom functions.\n\nCustom functions:\n\n- `glob` reports whether arg1 matches the shell file name pattern.\n- `re` reports whether arg1 contains any match of the regular expression pattern.\n\nAll these functions accepts two or more arguments, returning in effect:\n\n\u003e func(arg1, arg2) || func(arg1, arg3) || func(arg1, arg4) ...\n\n## Build\n\nBuild job creates configurations from targets.\n\nConfiguration is a list of build rules:\n\n```yaml\n- \u003cbuild_rule_config\u003e\n```\n\nBuild rule configuration options:\n\n```yaml\n# Mandatory. Routes targets to this rule with tags matching this selector.\nselector: \u003cselector\u003e\n\n# Mandatory. Tags to add to all built by this rule configurations.\ntags: \u003ctags\u003e\n\n# Mandatory. Apply rules, at least one should be defined. \napply:\n  # Mandatory. Routes targets to this apply rule with tags matching this selector.\n  - selector: \u003cselector\u003e\n\n    # Optional. Tags to add to configurations built by this apply rule.\n    tags: \u003ctags\u003e\n\n    # Mandatory. Configuration template.\n    template: \u003ctemplate\u003e\n```\n\nTemplate syntax is [go-template](https://golang.org/pkg/text/template/).\n\n### Available functions\n\n- go-template [built-in functions](https://golang.org/pkg/text/template/#hdr-Functions).\n- [sprig functions](http://masterminds.github.io/sprig/).\n- custom functions.\n\nCustom functions:\n\n- `glob` reports whether arg1 matches the shell file name pattern.\n- `re` reports whether arg1 contains any match of the regular expression pattern.\n\nAll these functions accepts two or more arguments, returning in effect:\n\n\u003e func(arg1, arg2) || func(arg1, arg3) || func(arg1, arg4) ...\n\n## Export\n\nExport job exports configurations built by [build job](#Build).\n\nSupported exporters:\n\n- `file`\n\nExport configuration:\n\n```yaml\nfile:\n  - \u003cfile_exporter_config\u003e\n```\n\n### File\n\nFile exporter writes configurations to a specific file.\n\n```yaml\n# Mandatory. Routes configurations to this exporter with tags matching this selector.\nselector: \u003cselector\u003e\n\n# Mandatory. Absolute path to a file.\nfilename: \u003cfilename\u003e\n```\n\n## Troubleshooting\n\nService-discovery has debug mode and special `stdout` exporter which is enabled only when it's running from the\nterminal.\n\nCLI:\n\n```cmd\nUsage:\n  sd [OPTION]...\n\nApplication Options:\n      --config-file= Configuration file path\n      --config-map=  Configuration ConfigMap (name:key)\n  -d, --debug        Debug mode\n\nHelp Options:\n  -h, --help         Show this help message\n```\n\n\u003c/details\u003e","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnetdata%2Fagent-service-discovery","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnetdata%2Fagent-service-discovery","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnetdata%2Fagent-service-discovery/lists"}