https://github.com/percona/telemetry-agent
https://github.com/percona/telemetry-agent
Last synced: 7 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/percona/telemetry-agent
- Owner: percona
- License: agpl-3.0
- Created: 2023-09-25T13:03:41.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2025-07-15T16:01:20.000Z (12 months ago)
- Last Synced: 2025-07-15T16:50:44.080Z (12 months ago)
- Language: Go
- Size: 457 KB
- Stars: 2
- Watchers: 28
- Forks: 5
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Percona Telemetry
[](https://github.com/percona/telemetry-agent/actions/workflows/ci.yml)
[](https://cla-assistant.percona.com/percona/telemetry-agent)
[](https://goreportcard.com/report/github.com/percona/telemetry-agent)
* [Percona Telemetry](#percona-telemetry)
* [Installation-time telemetry](#installation-time-telemetry)
* [Disable installation-time telemetry](#disable-installation-time-telemetry)
* [Continuous telemetry](#continuous-telemetry)
* [Elements of the continuous telemetry system](#elements-of-the-continuous-telemetry-system)
* [Locations of metrics files and telemetry history](#locations-of-metrics-files-and-telemetry-history)
* [Metrics file format](#metrics-file-format)
* [Percona Telemetry Agent](#percona-telemetry-agent)
* [Telemetry agent payload example](#telemetry-agent-payload-example)
* [Telemetry Agent configuration](#telemetry-agent-configuration)
* [Disable continuous telemetry](#disable-continuous-telemetry)
* [Disable the Telemetry Agent](#disable-the-telemetry-agent)
* [Disable temporarily](#disable-temporarily)
* [Disable permanently](#disable-permanently)
* [Disable DB component](#disable-db-component)
This repository contains the Percona Telemetry Agent for Pillars telemetry project, a tool that collects telemetry
information and sends it to Percona Telemetry service.
For the time being Pillars (Percona Server for MySQL, Percona Server for MongoDB, Percona Server for PostgreSQL)
Telemetry consists of the following parts:
* [Installation-time telemetry](#installation-time-telemetry)
* [Continuous telemetry](#continuous-telemetry)
All telemetry data is sent to the Percona Telemetry service. The service is responsible for storing and processing the
data. The data is used to improve the quality of Percona products and services.
Telemetry data is sent to https://check.percona.com/v1/telemetry/GenericReport endpoint in a JSON format.
## Installation-time telemetry
This feature is enabled in Percona Linux packages and Docker containers by default. In order to disable it see
[Disable installation-time telemetry](#disable-installation-time-telemetry)
This telemetry is executed only once during software installation. It collects information at the moment of installation
or Docker container startup and does not run afterward.
For this purpose the script `call-home.sh` is used that collects telemetry information and sends it to a Percona Telemetry service.
The script validates only if mandatory parameters are provided. It does not check the validity and itegritiy of provided parameters.
Usage:
```{.bash data-prompt="$"}
./call-home.sh OPTIONS
OPTIONS can be:
-h Show this message
-f [PERCONA_PRODUCT_FAMILY] Product family identifier. [REQUIRED]
-v [PERCONA_PRODUCT_VERSION] Product version. [REQUIRED]
-s [PERCONA_OPERATING_SYSTEM] Operating system identifier. [Default: autodetected with fallback to "unknown"]
-d [PERCONA_DEPLOYMENT_METHOD] Deployment method. [REQUIRED]
-i [PERCONA_INSTANCE_ID] Instance id [Default: autogenerated]
-j [PERCONA_TELEMETRY_CONFIG_FILE_PATH] Path of the file where to store the unique ID of this node. [Default: /usr/local/percona/telemetry_uuid]
-u [PERCONA_TELEMETRY_URL] Percona Telemetry Service endpoint [Default: https://check.percona.com/v1/telemetry/GenericReport]
-c [PERCONA_CONNECT_TIMEOUT] Default timeout for the curl to establish connection. [Default: 5]
-t [PERCONA_SEND_TIMEOUT] Default timeout for the whole curl command. [Default: 10]
```
Note that `-d PERCONA_PRODUCT_FAMILY` can be set to any string, but only the following ones will be accepted
by Percona Telemetry service (there is no validation of the script side):
```
PRODUCT_FAMILY_PS
PRODUCT_FAMILY_PXC
PRODUCT_FAMILY_PXB
PRODUCT_FAMILY_PSMDB
PRODUCT_FAMILY_PBM
PRODUCT_FAMILY_POSTGRESQL
PRODUCT_FAMILY_PMM
PRODUCT_FAMILY_EVEREST
PRODUCT_FAMILY_PERCONA_TOOLKIT
```
For example, on Debian-derived distribution, you may run the script as:
```{.bash data-prompt="$"}
./call-home.sh -f "PRODUCT_FAMILY_PS" -v "8.0.33" -s "$(cat /etc/issue)" -i "13f5fc62-35b4-4716-b3e6-96c761fc204d" -j /tmp/percona.telemetry -u https://check.percona.com/v1/telemetry/GenericReport -c 1 -t 2
```
on a Red Hat-derived distribution, you may run the script as:
```{.bash data-prompt="$"}
./call-home.sh -f "PRODUCT_FAMILY_PS" -v "8.0.33" -s "$(cat /etc/redhat-release)" -i "13f5fc62-35b4-4716-b3e6-96c761fc204d" -j /tmp/percona.telemetry -u https://check.percona.com/v1/telemetry/GenericReport -c 1 -t 2
```
### Disable installation-time telemetry
The data collection may be disabled via setting an environment variable `PERCONA_TELEMETRY_DISABLE=1` during Linux package
installation or Docker container startup.
For example, on Debian-derived distribution (this action does not affect the continuous telemetry part):
```{.bash data-prompt="$"}
sudo PERCONA_TELEMETRY_DISABLE=1 apt install percona-server-server
```
on a Red Hat-derived distribution (this action does not affect the continuous telemetry part):
```{.bash data-prompt="$"}
sudo PERCONA_TELEMETRY_DISABLE=1 dnf install percona-server-server
```
or in a Docker container (this action disables the continuous telemetry part as well):
```{.bash data-prompt="$"}
docker run -e PERCONA_TELEMETRY_DISABLE=1 percona/percona-server
```
## Continuous telemetry
This telemetry part involves setting up a Telemetry Agent and a database (DB) component. It continuously collects and
sends information daily.
The Telemetry Agent runs at scheduled daily intervals to collect data. The agent gathers data (for example, usage statistics)
and sends this information to the Percona Telemetry Service.
### Elements of the continuous telemetry system
Percona collects information using these elements:
| Function | Description |
|-------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Percona Telemetry DB component | This component collects metrics directly from the database and stores them in a metrics file. |
| [Metrics File](#locations-of-metrics-files-and-telemetry-history) | This standalone file on the database host's file system stores the collected metrics. |
| [Telemetry Agent](#percona-telemetry-agent) | This independent process runs on your database host's operating system and performs the following tasks:
- Collects OS-level metrics.
- Reads the metrics file and adds the OS-level metrics.
- Sends the complete set of metrics to the Percona Platform.
- Collects the list of installed Percona packages using the local package manager. |
### Locations of metrics files and telemetry history
Percona stores the Metrics file in one of the following directories on the local file system. The location depends on the product.
* Telemetry root path - `/usr/local/percona/telemetry`
* PSMDB (mongod) root path - `${telemetry root path}/psmdb/`
* PSMDB (mongos) root path - `${telemetry root path}/psmdbs/`
* PS root path - `${telemetry root path}/ps/`
* PXC root path - `${telemetry root path}/pxc/`
* PG root path - `${telemetry root path}/pg/`
Percona archives the telemetry history in `${telemetry root path}/history/`.
### Metrics file format
The Metrics file uses the Javascript Object Notation (JSON) format. Percona reserves the right to extend the current set
of JSON structure attributes in the future.
An example of the Metrics file content is the following:
```json
{
"db_instance_id": "e83c568c-e140-11ee-8320-7e207666b18a",
"pillar_version": "8.0.35-27",
"active_plugins": [
"binlog",
"mysql_native_password",
"sha256_password",
"caching_sha2_password",
"sha2_cache_cleaner",
"daemon_keyring_proxy_plugin",
"PERFORMANCE_SCHEMA",
"ROCKSDB_INDEX_FILE_MAP",
"ROCKSDB_LOCKS",
"ROCKSDB_TRX",
"ROCKSDB_DEADLOCK"
],
"active_components": [
"file://component_percona_telemetry"
],
"uptime": "6185",
"databases_count": "7",
"databases_size": "33149",
"se_engines_in_use": [
"InnoDB",
"ROCKSDB"
],
"replication_info": {
"is_semisync_source": "1",
"is_replica": "1"
}
}
```
### Percona Telemetry Agent
This program, called `percona-telemetry-agent`, constantly runs in the background on your server's host system.
It manages JSON files, which store the collected data in a specific location (`${telemetry root path}`). This agent can
create, read, write, and delete these files.
The agent's log file, containing information about its activity, is located at `/var/log/percona/telemetry-agent.log`.
In the first 24 hours, no information is collected or sent. After that period, the agent tries to send the collected
information to Percona Telemetry Service daily. If this operation fails, the agent retries up to five times.
After the data is successfully sent, the agent saves a copy of the sent data in a separate "history" folder
(`${telemetry root path}/history/`), and then, deletes the original file created by the database.
The agent won't send any data if the target directory doesn't contain specific files related to Percona software.
#### Telemetry agent payload example
The following is an example of a Telemetry Agent payload:
```json
{
"reports": [
{
"id": "B5BDC47B-B717-4EF5-AEDF-41A17C9C18BB",
"createTime": "2023-09-01T10:56:49Z",
"instanceId": "B5BDC47B-B717-4EF5-AEDF-41A17C9C18BA",
"productFamily": "PRODUCT_FAMILY_PS",
"metrics": [
{
"key": "OS",
"value": "Ubuntu"
},
{
"key": "pillar_version",
"value": "8.0.33-25"
}
]
}
]
}
```
The agent sends information about the database and metrics.
| Key | Description |
|---------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| id | A randomly generated Universally Unique Identifier (UUID) version 4 of request |
| createTime | UNIX timestamp of request creation |
| instanceId | The DB Host ID. The value can be taken from the `instanceId`, the `/usr/local/percona/telemetry_uuid`
or generated as a UUID version 4 if the file is absent |
| productFamily | The value from the file path |
| metrics | An array of key:value pairs collected from the Metrics file |
The following operating system-level metrics are sent with each metrics send iteration:
| Key | Description |
|----------------------|--------------------------------------------------------------------------------------------|
| "OS" | The name of the operating system |
| "hardware_arch" | CPU architecture used on DB host |
| "deployment" | How the application was deployed.
The possible values could be "PACKAGE" or "DOCKER". |
| "installed_packages" | A list of the installed Percona's packages. |
#### Telemetry Agent configuration
Telemetry Agent can be configured during startup by setting the following environment variables or their CLI arguments equivalents:
| Environment variable | CLI param | Description | Default value |
|-----------------------------------------|-----------------------------------|-----------------------------------------------------------------|------------------------------------------------------|
| PERCONA_TELEMETRY_ROOT_PATH | --telemetry.root-path | The root path for telemetry files | /usr/local/percona/telemetry |
| PERCONA_TELEMETRY_CHECK_INTERVAL | --telemetry.check-interval | The interval in seconds between telemetry checks | 86400 |
| PERCONA_TELEMETRY_RESEND_INTERVAL | --telemetry.resend-interval | The interval in seconds between telemetry resend attempts | 60 |
| PERCONA_TELEMETRY_HISTORY_KEEP_INTERVAL | --telemetry.history-keep-interval | The interval in seconds between telemetry history files cleanup | 604800 |
| PERCONA_TELEMETRY_URL | --telemetry.url | The URL of the Percona Telemetry Service | https://check.percona.com/v1/telemetry/GenericReport |
| | --log.verbose | Enable verbose logging | false |
| | --log.dev-mode | Enable development mode | false |
| | --version | Print version and exit | false |
| | --help | Show help | false |
Changing any of this configuration parameters requires a restart of the Telemetry Agent.
### Disable continuous telemetry
Percona software enables the continuous telemetry system by default. Disable the Telemetry agent and uninstall the DB
component to turn off this telemetry completely.
These actions do not affect [Installation-time telemetry](#installation-time-telemetry).
#### Disable the Telemetry Agent
You can either disable the Telemetry agent for a session or permanently. These actions do not affect the DB component, i.e.
DB will still continue collecting metrics but they will not be sent to Percona Telemetry Service.
##### Disable temporarily
Turn off Telemetry Agent temporarily until the next server restart:
```{.bash data-prompt=$}
systemctl stop percona-telemetry-agent
```
##### Disable permanently
Turn off Telemetry Agent permanently:
```{.bash data-prompt=$}
systemctl stop percona-telemetry-agent
systemctl disable percona-telemetry-agent
```
#### Disable DB component
The DB component continues to generate daily telemetry files and store them for a week, even after you stop the
Telemetry Agent service.
Refer to particular Percona DB server documentation in order to disable the DB component.