https://github.com/netdata/agent-service-discovery
https://github.com/netdata/agent-service-discovery
Last synced: 10 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/netdata/agent-service-discovery
- Owner: netdata
- License: mit
- Archived: true
- Created: 2020-05-06T09:29:03.000Z (almost 6 years ago)
- Default Branch: master
- Last Pushed: 2025-01-30T10:07:12.000Z (about 1 year ago)
- Last Synced: 2025-02-27T03:05:03.551Z (about 1 year ago)
- Language: Go
- Size: 313 KB
- Stars: 15
- Watchers: 11
- Forks: 10
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project
README
# Service discovery
> [!WARNING]
>
> **Deprecation Notice**: This repository's service discovery functionality has been migrated to go.d.plugin in the main [Netdata repository](https://github.com/netdata/netdata). All future development, maintenance, and updates will continue there.
Old readme
Service discovery extracts all the potentially useful information from different sources, converts it to the
configurations and exports them to the different destinations.
## Pipeline
The service discovery pipeline has four jobs:
| Job | Description |
|:-----------------------:|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [discovery](#Discovery) | Dynamically discovers monitoring targets by collecting events from kubernetes API server. It collects POD and SERVICE events. |
| [tag](#Tag) | Dynamically add tags to discovered monitoring targets. Based on the POD and SERVICE fields and using patterns on them, one or more tags are attached to the monitoring targets. |
| [build](#Build) | Dynamically creates data collection configurations for the monitored targets, using templates. |
| [export](#Export) | Dynamically exports data collection configurations to allow netdata data collection plugins to use them. Data collection jobs in netdata are created and destroyed as needed. |
Routing in a job and between jobs based on `tags` and `selector`.
Pipeline configuration:
```yaml
name:
discovery:
tag:
build:
export:
```
## Tags and selectors
Tag, build and export jobs have `selector`, the pipeline routes a target/config to the job only if its tags matches job
selectors.
Both tags and selector are just lists of words.
A word must match the regex `^[a-zA-Z][a-zA-Z0-9=_.]*$`.
Tags special cases:
- `-word`: the word will be removed on tags merging.
Selectors special cases:
- `!word`: shouldn’t contain the word.
- `word|word|word`: should contain any word.
## Discovery
Discovery job dynamically discovers targets using one of the supported service-discovery mechanisms.
Supported mechanisms:
- [kubernetes](#Kubernetes)
Discovery configuration:
```yaml
k8s:
-
```
### Kubernetes
Kubernetes discoverer retrieves targets from [Kubernetes'](https://kubernetes.io/)
[REST API](https://kubernetes.io/docs/reference/). It always stays synchronized with the cluster state.
Configuration options:
```yaml
# Mandatory. Tags to add to all discovered targets.
tags:
# Mandatory. The Kubernetes role of entities that should be discovered.
role:
# Optional. Discover only targets that exist on the same node as service-discovery.
# This option works only for 'pod' role and it requires MY_NODE_NAME env variable to be set.
local_mode:
# Optional. If omitted, all namespaces are used.
namespaces:
-
```
One of the following role types can be configured to discover targets:
- `pod`
- `service`
#### Pod Role
The pod role discovers all pods and exposes their containers as targets. For each declared port of a container, it
generates single target. If there is no declared port it generates one target with empty `Port`, `PortName`
and `PortProtocol` fields.
Available pod target fields:
| Name | Type | Value |
|:-----------------|:------------------|:----------------------------------------------------------|
| `TUID` | string | `Namespace_Name_ContName_PortProtocol_Port` |
| `Address` | string | `PodIP:Port` |
| `Namespace` | string | _pod.metadata.namespace_ |
| `Name` | string | _pod.metadata.name_ |
| `Annotations` | map[string]string | _pod.metadata.annotations_ |
| `Labels` | map[string]string | _pod.metadata.labels_ |
| `NodeName` | string | _pod.spec.nodeName_ |
| `PodIP` | string | _pod.status.podIP_ |
| `ControllerName` | string | _pod.OwnerReferences.Controller.Name_ |
| `ControllerKind` | string | _pod.OwnerReferences.Controller.Kind_ |
| `ContName` | string | _pod.spec.containers.name_ |
| `Image` | string | _pod.spec.containers.image_ |
| `Env` | map[string]string | _pod.spec.containers.env_ + _pod.spec.containers.envFrom_ |
| `Port` | string | _pod.spec.containers.ports.containerPort_ |
| `PortName` | string | _pod.spec.containers.ports.name_ |
| `PortProtocol` | string | _pod.spec.containers.ports.protocol_ |
#### Service Role
The service role discovers a target for each service port for each service.
Available service target fields:
| Name | Type | Value |
|:---------------|:------------------|:------------------------------------------|
| `TUID` | string | `Namespace_Name_PortProtocol_Port` |
| `Address` | string | `Name.Namespace.svc:Port` |
| `Namespace` | string | _svc.metadata.namespace_ |
| `Name` | string | _svc.metadata.name_ |
| `Annotations` | map[string]string | _svc.metadata.annotations_ |
| `Labels` | map[string]string | _svc.metadata.labels_ |
| `Port` | string | _pod.spec.containers.ports.containerPort_ |
| `PortName` | string | _pod.spec.containers.ports.name_ |
| `PortProtocol` | string | _pod.spec.containers.ports.protocol_ |
| `ClusterIP` | string | _svc.spec.clusterIP_ |
| `ExternalName` | string | _svc.spec.externalName_ |
| `Type` | string | _svc.spec.ports.type_ |
## Tag
Tag job tags targets discovered by [discovery job](#Discovery). Its purpose is service identification.
Configuration is a list of tag rules:
```yaml
-
```
Tag rule configuration options:
```yaml
# Mandatory. Routes targets to this tag rule with tags matching this selector.
selector:
# Mandatory. Tags to merge with the target tags if at least on of the match rules matches.
tags:
# Mandatory. Match rules, at least one should be defined.
match:
# Optional. Routes targets to this match rule with tags matching this selector.
- selector:
# Mandatory. Tags to merge with the target tags if this rule expression evaluates to true.
tags:
# Mandatory. Match expression.
expr:
```
**Match expression evaluation result should be true or false**.
Expression syntax is [go-template](https://golang.org/pkg/text/template/).
### Available functions
- go-template [built-in functions](https://golang.org/pkg/text/template/#hdr-Functions).
- [sprig functions](http://masterminds.github.io/sprig/).
- custom functions.
Custom functions:
- `glob` reports whether arg1 matches the shell file name pattern.
- `re` reports whether arg1 contains any match of the regular expression pattern.
All these functions accepts two or more arguments, returning in effect:
> func(arg1, arg2) || func(arg1, arg3) || func(arg1, arg4) ...
## Build
Build job creates configurations from targets.
Configuration is a list of build rules:
```yaml
-
```
Build rule configuration options:
```yaml
# Mandatory. Routes targets to this rule with tags matching this selector.
selector:
# Mandatory. Tags to add to all built by this rule configurations.
tags:
# Mandatory. Apply rules, at least one should be defined.
apply:
# Mandatory. Routes targets to this apply rule with tags matching this selector.
- selector:
# Optional. Tags to add to configurations built by this apply rule.
tags:
# Mandatory. Configuration template.
template:
```
Template syntax is [go-template](https://golang.org/pkg/text/template/).
### Available functions
- go-template [built-in functions](https://golang.org/pkg/text/template/#hdr-Functions).
- [sprig functions](http://masterminds.github.io/sprig/).
- custom functions.
Custom functions:
- `glob` reports whether arg1 matches the shell file name pattern.
- `re` reports whether arg1 contains any match of the regular expression pattern.
All these functions accepts two or more arguments, returning in effect:
> func(arg1, arg2) || func(arg1, arg3) || func(arg1, arg4) ...
## Export
Export job exports configurations built by [build job](#Build).
Supported exporters:
- `file`
Export configuration:
```yaml
file:
-
```
### File
File exporter writes configurations to a specific file.
```yaml
# Mandatory. Routes configurations to this exporter with tags matching this selector.
selector:
# Mandatory. Absolute path to a file.
filename:
```
## Troubleshooting
Service-discovery has debug mode and special `stdout` exporter which is enabled only when it's running from the
terminal.
CLI:
```cmd
Usage:
sd [OPTION]...
Application Options:
--config-file= Configuration file path
--config-map= Configuration ConfigMap (name:key)
-d, --debug Debug mode
Help Options:
-h, --help Show this help message
```