{"id":26576515,"url":"https://github.com/getsentry/statsdproxy","last_synced_at":"2025-03-23T03:21:38.616Z","repository":{"id":190490270,"uuid":"681184765","full_name":"getsentry/statsdproxy","owner":"getsentry","description":"A proxy for transforming, pre-aggregating and routing statsd metrics","archived":false,"fork":false,"pushed_at":"2025-01-24T10:27:18.000Z","size":114,"stargazers_count":5,"open_issues_count":3,"forks_count":1,"subscribers_count":32,"default_branch":"main","last_synced_at":"2025-03-22T06:47:11.245Z","etag":null,"topics":["hackweek","metrics","statsd","tag-non-production"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/getsentry.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"custom":["https://sentry.io/pricing/","https://sentry.io/"]}},"created_at":"2023-08-21T13:04:36.000Z","updated_at":"2024-12-17T22:09:22.000Z","dependencies_parsed_at":"2025-01-22T12:22:28.768Z","dependency_job_id":"52983a70-b2cd-49b7-8aa8-88100024c346","html_url":"https://github.com/getsentry/statsdproxy","commit_stats":null,"previous_names":["getsentry/statsdproxy"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/getsentry%2Fstatsdproxy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/getsentry%2Fstatsdproxy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/getsentry%2Fstatsdproxy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/getsentry%2Fstatsdproxy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/getsentry","download_url":"https://codeload.github.com/getsentry/statsdproxy/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244918586,"owners_count":20531686,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hackweek","metrics","statsd","tag-non-production"],"created_at":"2025-03-23T03:21:37.479Z","updated_at":"2025-03-23T03:21:38.582Z","avatar_url":"https://github.com/getsentry.png","language":"Rust","funding_links":["https://sentry.io/pricing/","https://sentry.io/"],"categories":[],"sub_categories":[],"readme":"# statsdproxy\n\nA proxy for transforming, pre-aggregating and routing statsd metrics, like\n[Veneur](https://github.com/stripe/veneur), [Vector](https://vector.dev/) or\n[Brubeck](https://github.com/github/brubeck).\n\nCurrently supports the following transformations:\n\n* Deny- or allow-listing of specific tag keys or metric names\n* Adding hardcoded tags to all metrics\n* Basic cardinality limiting, tracking the number of distinct tag values per\n  key or the number of overall timeseries (=combinations of metrics and tags).\n\nSee `example.yml` for details.\n\nA major goal is minimal overhead and **no loss of information** due to\nunnecessarily strict parsing. Statsdproxy intends to orient itself around\n[dogstatsd](https://docs.datadoghq.com/developers/dogstatsd/datagram_shell/?tab=metrics)\nprotocol but should gracefully degrade for other statsd dialects, in that those\nmetrics and otherwise unparseable bytes will be forwarded as-is.\n\n**This is not a Sentry product**, not deployed in any sort of production\nenvironment, but a side-project done during Hackweek.\n\n\n## Basic usage\n\n1. Run a \"statsd server\" on port 8081 that just prints metrics\n\n   ```\n   socat -u UDP-RECVFROM:8081,fork SYSTEM:\"cat; echo\"\n   ```\n\n2. Copy `example.yaml` to `config.yaml` and edit it\n3. Run statsdproxy to read metrics from port 8080, transform them using the\n   middleware in `config.yaml` and forward the new metrics to port 8081:\n\n   ```\n   cargo run --release -- --listen 127.0.0.1:8080 --upstream 127.0.0.1:8081 -c config.yaml\n   ```\n\n5. Send metrics to statsdproxy:\n\n   ```\n   yes 'users.online:1|c|@0.5' | nc -u 127.0.0.1 8080\n   ```\n\n4. You should see new metrics in `socat` with your middlewares applied.\n\n## Usage with Snuba\n\nPatch the following settings in `snuba/settings/__init__.py`:\n\n```python\nDOGSTATSD_HOST = \"127.0.0.1\"\nDOGSTATSD_PORT = \"8080\"\n```\n\nThis will send metrics to port 8080.\n\n## Processing model\n\nThis is the processing model used by the provided server. It should be respected\nby any usage of this software as a library.\n\n* The server receives metrics as bytes over udp, either singly or several joined\n  with `\\n`.\n* For every metric received, the server invokes the `poll` method of the topmost\n  middleware.\n    * The middleware may use this invocation to do any needed internal\n      bookkeeping.\n    * The middleware should then invoke the `poll` method of the next\n      middleware, if any.\n* Once `poll` returns, the server invokes the `submit` method of the topmost\n  middleware with a mutable reference to the current metric.\n    * The middleware should process the metric.\n        * If processing was successful, and if appropriate to its function\n          (eg. a metric aggregator might hold onto metrics), the middleware\n          should `submit` the processed metric to the next middleware, returning\n          the result of this call.\n        * If processing was unsuccessful (eg. unknown StatsD dialect), the\n          unchanged metric should be treated as the processed metric, and passed\n          on or held as above.\n        * If a middleware becomes unable to handle more metrics during\n          processing, such that it cannot handle the current metric, it should\n          return `Overloaded`.\n    * If an overload is indicated, the server shall pause (TODO: how long)\n      before calling `submit` again with the same metric. (If an overload is\n      indicated too many times, maybe drop the metric?)\n* Separately, if no metric is received by the server for 1 second, it will\n  invoke the `poll` method of the topmost middleware. This invocation of `poll`\n  should be handled the same as above.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgetsentry%2Fstatsdproxy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgetsentry%2Fstatsdproxy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgetsentry%2Fstatsdproxy/lists"}