https://github.com/influxdata/influxdb-observability
https://github.com/influxdata/influxdb-observability
Last synced: 7 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/influxdata/influxdb-observability
- Owner: influxdata
- License: mit
- Created: 2021-03-12T01:36:18.000Z (almost 5 years ago)
- Default Branch: main
- Last Pushed: 2025-03-28T12:08:53.000Z (9 months ago)
- Last Synced: 2025-04-07T13:00:44.568Z (9 months ago)
- Language: Go
- Size: 1.8 MB
- Stars: 50
- Watchers: 12
- Forks: 27
- Open Issues: 12
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project
README
# InfluxDB Observability
This repository is a reference for converting observability signals (traces, metrics, logs) to/from a common InfluxDB schema.
## Demo
Steps to run the write full write/query demo follow.
In an InfluxDB Cloud 2 account backed by IOx, create a bucket named `otel`.
Then, create a token with permission to read and write to that bucket.
In demo/docker-compose.yml, set values for these keys.
The key `INFLUXDB_BUCKET_ARCHIVE` is optional;
if set, it should point to an InfluxDB bucket with longer retention policy than `INFLUXDB_BUCKET`,
so that the "Archive Trace" button in Jaeger works properly:
```yaml
INFLUXDB_ADDR:
INFLUXDB_BUCKET: otel
INFLUXDB_BUCKET_ARCHIVE: otel-archive
INFLUXDB_TOKEN:
```
In demo/otelcol-config.yml, set the similar values for these keys:
```yaml
endpoint: https://< region specific URL - https://region.csp.cloud2.influxdata.com/ >
bucket: otel
token:
```
Run the docker compose:
```console
$ docker compose --file demo/docker-compose.yml --project-directory . up --abort-on-container-exit --remove-orphans
```
Traces are generated by "HotRod", an application designed to demonstrate tracing.
Browse to HotRod at http://localhost:8080 and click some buttons to trigger trace activity.
Query those traces.
Browse to Jaeger at http://localhost:16686 and click "Find Traces" near the bottom left.
Click any trace.
View the dependency graph.
Click "System Architecture".
The images `otelcol-influxdb` and `jaeger-influxdb` are automatically built and pushed to Docker at https://hub.docker.com/r/jacobmarble/otelcol-influxdb and https://hub.docker.com/r/jacobmarble/jaeger-influxdb .
## Schema Reference
[Schema reference with conversion tables](docs/index.md).
## Modules
### `common`
The golang package `common` contains simple utilities and common string values,
used in at least two of the above-mentioned packages.
### `otel2influx` and `influx2otel`
The golang package [`otel2influx`](otel2influx/README.md) converts OpenTelemetry protocol buffer objects to (measurement, tags, fields, timestamp) tuples.
It is imported by [the OpenTelemetry Collector InfluxDB exporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/influxdbexporter)
and by [the Telegraf OpenTelemetry input plugin](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/opentelemetry).
The golang package [`influx2otel`](influx2otel/README.md) converts (measurement, tags, fields, timestamp) tuples to OpenTelemetry protocol buffer objects.
It is imported by [the OpenTelemtry Collector InfluxDB receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/influxdbreceiver)
and by [the Telegraf OpenTelemetry output plugin](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/opentelemetry).
### `jaeger-influxdb`
The [Jaeger Query Plugin for InfluxDB](jaeger-influxdb) enables querying traces stored in InfluxDB/IOx via the Jaeger UI.
### `tests-integration`
The golang package `tests-integration` contains integration tests.
These tests exercise the above packages against OpenTelemetry Collector Contrib and Telegraf.
To run these tests:
```console
$ cd tests-integration
$ go test
```
## Development
The file `demo/docker-compose-development.yml` is similar to the demo above,
but it uses local builds of the `otelcol-influxdb` and `jaeger-influxdb` images.
Build the needed docker images:
```console
$ docker compose --file demo/docker-compose-development.yml --project-directory . build
```
Start the development demo environment:
```console
$ docker compose --file demo/docker-compose-development.yml --project-directory . up --abort-on-container-exit --remove-orphans
```
## Contributing
Changes can be tested on a local branch using the `run-checks.sh` tool.
`run-checks.sh` verifies `go mod tidy` using `git diff`,
so any changes must be staged for commit in order for `run-checks.sh` to pass.
To update critical dependencies (OpenTelemetry, Jaeger, and intra-repo modules) in the various modules of this repository:
- run `update-deps.sh`
- stage the changed `go.mod` and `go.sum` files
- run `run-checks.sh`
## TODO
Fork this demo:
https://github.com/open-telemetry/opentelemetry-demo