{"id":21293111,"url":"https://github.com/criteo/graphite-remote-adapter","last_synced_at":"2025-07-11T16:32:18.470Z","repository":{"id":21517140,"uuid":"93149529","full_name":"criteo/graphite-remote-adapter","owner":"criteo","description":"Fully featured graphite remote adapter for Prometheus","archived":false,"fork":false,"pushed_at":"2022-12-02T04:51:08.000Z","size":7277,"stargazers_count":36,"open_issues_count":12,"forks_count":25,"subscribers_count":15,"default_branch":"master","last_synced_at":"2024-06-20T14:30:14.210Z","etag":null,"topics":["graphite","prometheus","storage"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/criteo.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-06-02T09:22:01.000Z","updated_at":"2024-06-20T14:30:14.211Z","dependencies_parsed_at":"2023-01-12T08:45:56.713Z","dependency_job_id":null,"html_url":"https://github.com/criteo/graphite-remote-adapter","commit_stats":null,"previous_names":[],"tags_count":24,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/criteo%2Fgraphite-remote-adapter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/criteo%2Fgraphite-remote-adapter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/criteo%2Fgraphite-remote-adapter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/criteo%2Fgraphite-remote-adapter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/criteo","download_url":"https://codeload.github.com/criteo/graphite-remote-adapter/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225741166,"owners_count":17516895,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["graphite","prometheus","storage"],"created_at":"2024-11-21T13:53:19.990Z","updated_at":"2024-11-21T13:53:20.681Z","avatar_url":"https://github.com/criteo.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Graphite Remote storage adapter [![Build Status](https://travis-ci.org/criteo/graphite-remote-adapter.svg?branch=master)](https://travis-ci.org/criteo/graphite-remote-adapter)\n\nThis is a read/write adapter that receives samples via Prometheus's remote write\nprotocol and stores them in remote storage like Graphite.\n\nIt is based on [remote_storage_adapter](https://github.com/prometheus/prometheus/tree/master/documentation/examples/remote_storage/remote_storage_adapter)\n\n### Compiling the binary\n\nYou can either `go get` it:\n\n```\n$ go get -d github.com/criteo/graphite-remote-adapter/...\n$ cd $GOPATH/src/github.com/criteo/graphite-remote-adapter\n$ make build\n$ ./graphite-remote-adapter --graphite.read.url='http://localhost:8080' --graphite.write.carbon-address=localhost:2003\n```\n\nOr checkout the source code and build manually:\n\n```\n$ mkdir -p $GOPATH/src/github.com/criteo\n$ cd $GOPATH/src/github.com/criteo\n$ git clone https://github.com/criteo/graphite-remote-adapter.git\n$ cd graphite-remote-adapter\n$ make build\n$ ./graphite-remote-adapter --graphite.read.url='http://localhost:8080' --graphite.write.carbon-address=localhost:2003\n```\n\n## Running\n\nGraphite example:\n\n```\n./graphite-remote-adapter \\\n  --graphite.write.carbon-address=localhost:2001 \\\n  --graphite.read.url='http://guest:guest@localhost:8080' \\\n  --read.timeout=10s --write.timeout=5s \\\n  --read.delay 3600s \\\n  --graphite.default-prefix prometheus.\n```\n\nTo show all flags:\n\n```\n./graphite-remote-adapter -h\n```\n\n## Example\nYou can provide some configuration parameters either as flags or in a configuration file. If defined in both, the flag is used.\nIn addtion, you can fill the configuration file with Graphite specific parameters. You can indeed defined customized paths/behaviors for remote-write into Graphite.\n\nThis is an example configuration that should cover most relevant aspects of the YAML configuration format.\n\n```yaml\nweb:\n  listen_address: \"0.0.0.0:9201\"\n  telemetry_path: \"/metrics\"\nwrite:\n  timeout: 5m\nread:\n  timeout: 5m\n  delay: 1h\n  ignore_error: true\ngraphite:\n  default_prefix: test.prefix.\n  enable_tags: false\n  read:\n    url: http://localhost:8888\n  write:\n    carbon_address: localhost:2003\n    carbon_transport: tcp\n    carbon_reconnect_interval: 5m\n    enable_paths_cache: true\n    paths_cache_ttl: 1h\n    paths_cache_purge_interval: 2h\n    template_data:\n      var1:\n        foo: bar\n      var2: foobar\n\n    rules:\n    - match:\n        owner: team-X\n      match_re:\n        service: ^(foo1|foo2|baz)$\n      template: '{{.var1.foo}}.graphite.path.host.{{.labels.owner}}.{{.labels.service}}{{if ne .labels.env \"prod\"}}.{{.labels.env}}{{end}}'\n      continue: true\n    - match:\n        owner: team-X\n        env:   prod\n      template: 'bla.bla.{{.labels.owner | escape}}.great.{{.var2}}'\n      continue: true\n    - match:\n        owner: team-Z\n      continue: false\n\n```\n\n## Support for Tags\n\nGraphite 1.1.0 supports tags: http://graphite.readthedocs.io/en/latest/tags.html, you can\nenable support for tags in the remote adapter with `--graphite.enable-tags` or in the\nconfiguration file.\n\n### Filtering tags\n\nUsing `--graphite.filtered-tags` (or the `filtered_tags` yaml field in configuration files), it is possible to exports as tags only a given set of label names. Other labels/values won't be exported as tags, and will still be part of the metric name. This feature is only supported for Graphite Tags (not available when using the OpenMetrics format).\n\n## Configuring Prometheus\n\nTo configure Prometheus to send samples to this binary, add the following to your `prometheus.yml`:\n\n```yaml\n# Remote write configuration.\nremote_write:\n  - url: \"http://localhost:9201/write\"\n\n# Remote read configuration.\nremote_read:\n  - url: \"http://localhost:9201/read\"\n```\n\nSince the 0.0.15, a custom prefix can be set in the query string and this will replace the default one. This could be useful if you are using the graphite remote adapter for multiple Prometheus instances with different prefix.\n\n```yaml\n# Remote write configuration.\nremote_write:\n  - url: \"http://localhost:9201/write?graphite.default-prefix=customprefix.\"\n\n# Remote read configuration.\nremote_read:\n  - url: \"http://localhost:9201/read?graphite.default-prefix=customprefix.\"\n```\n\n## Testing\n\nYou can test the graphite-remote-adapter behavior or its configuration using the second binary named **ratool** for remote-adapter tool.\nHere are two examples:\n\n#### Integration test (manual end-to-end)\n\nThe remote-adapter tool will read an input file in Prometheus exposition text format;\ntranslate it in WriteRequest using compressed protobuf format; and send it to\nthe graphite-remote-adapter url on its /write endpoint.\nNo need to run a Prometheus instance to test it anymore:\n\nfile -\u003e ratool -\u003e graphite-remote-adapter -\u003e nc\n```\n$ make build\n$ cat cmd/ratool/input.metrics.example\n  # Use the Prometheus exposition text format\n  toto{foo=\"bar\", cluster=\"test\"} 42\n  toto{foo=\"bar\", cluster=\"canary\"} 34\n  # You can even force a given timestamp\n  toto{foo=\"bazz\", cluster=\"canary\"} 18 1528819131000\n$ ./graphite-remote-adapter --graphite.write.carbon-address ':8888' --log.level debug \u0026\n$ nc -l 0.0.0.0 8888 -w 1 \u003e out.txt\n$ ./ratool mock-write --metrics.file cmd/ratool/input.metrics.example --remote-adapter.url 'http://localhost:9201'\n$ cat out.txt\n  toto.cluster.test.foo.bar 42.000000 1570803131\n  toto.cluster.canary.foo.bar 34.000000 1570803131\n  toto.cluster.canary.foo.bazz 18.000000 1528819131\n```\n\n#### Unittests (automated config unittests)\n\nIf you want to unit test your configurations without requiring any network, define a file for each configuration you\nwant to test.\n\nExample:\n\n```yaml\nconfig_file: config.yml\ntests:\n  - name: \"Test label\"\n    input: |\n        # Use the Prometheus exposition text format\n        toto{foo=\"bar\", cluster=\"test\"} 42 1570802650000\n        toto{foo=\"bar\", cluster=\"canary\"} 34 1570802650000\n        toto{foo=\"bazz\", cluster=\"canary\"} 18 1528819131000\n    output: |\n        toto.my.templated.path.test.foo.bar.lulu 42.000000 1570802650\n        toto.canary.other.template.bar 34.000000 1570802650\n        toto.canary.other.template.bazz 18.000000 1528819131\n\n  - name: \"Other test\"\n    input: |\n        foo{bar=\"baz\"} 10\n    output: |\n        foo.bar.baz.lol 10 1528819131000\n```\n\nThe path to `config_file` is relative to the test file.\n\n\u003e *Note:* timestamps do not have the same unit for input and output. Input uses a regular unix timestamp in \n\u003e milliseconds, output is in seconds.\n\nTo run it:\n\n```\n$ make build\n$ ./ratool unittest --test.file test_file.yml\n```\n\nThe tool will exit with a non-zero code if the output of the remote adapter for the given configuration and the given \ninput does not match the expected output (order of the lines is not checked). \n\nIt also prints the diff on the standard error stream. \n\nExample of output:\n\n```plain\n./ratool unittest --config.file foo.yml --test.file bar.yml\n# Testing foo.yml\n## Test label\n-toto.my.templated.path.test.foo.bar.lulu 42.000000 1570802650\n-toto.canary.other.template.bar 34.000000 1570802650\n-toto.canary.other.template.bazz 18.000000 1528819131\n+toto.cluster.test.foo.bar 42.000000 1536658898\n+\n+toto.cluster.canary.foo.bar 34.000000 1536658898\n+\n+toto.cluster.canary.foo.bazz 18.000000 1528819131\n## Other test\n-foo.bar.baz.lol 10 1528819131000\n+foo.bar.baz 10.000000 1536658898\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcriteo%2Fgraphite-remote-adapter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcriteo%2Fgraphite-remote-adapter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcriteo%2Fgraphite-remote-adapter/lists"}