{"id":21187540,"url":"https://github.com/weisdd/lfgw","last_synced_at":"2025-07-10T02:31:09.215Z","repository":{"id":36971229,"uuid":"346500909","full_name":"weisdd/lfgw","owner":"weisdd","description":"A reverse proxy aimed at PromQL / MetricsQL metrics filtering based on OIDC roles","archived":false,"fork":false,"pushed_at":"2024-04-29T05:10:26.000Z","size":1604,"stargazers_count":18,"open_issues_count":7,"forks_count":5,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-06-19T02:06:52.281Z","etag":null,"topics":["go","golang","metrics","metricsql","prometheus","promql","reverse-proxy","victoriametrics"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/weisdd.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-03-10T21:46:58.000Z","updated_at":"2024-03-26T07:41:52.000Z","dependencies_parsed_at":"2023-02-18T03:31:35.082Z","dependency_job_id":"5fde96c0-f2dd-4b86-9529-14db178d0c0e","html_url":"https://github.com/weisdd/lfgw","commit_stats":{"total_commits":113,"total_committers":3,"mean_commits":"37.666666666666664","dds":0.2566371681415929,"last_synced_commit":"3a183ad0e9e6a9e50b3ce4c76d3b58af0e2e8906"},"previous_names":[],"tags_count":18,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/weisdd%2Flfgw","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/weisdd%2Flfgw/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/weisdd%2Flfgw/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/weisdd%2Flfgw/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/weisdd","download_url":"https://codeload.github.com/weisdd/lfgw/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225611651,"owners_count":17496507,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","golang","metrics","metricsql","prometheus","promql","reverse-proxy","victoriametrics"],"created_at":"2024-11-20T18:37:30.490Z","updated_at":"2024-11-20T18:37:31.148Z","avatar_url":"https://github.com/weisdd.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Label Filter Gateway (lfgw)\n\nlfgw is a simple reverse proxy aimed at PromQL / MetricsQL metrics filtering based on OIDC roles.\n\nIt relies on [VictoriaMetrics/metricsql](https://github.com/VictoriaMetrics/metricsql) for label filters manipulation in metric expressions (according to an [ACL](#acl)) before a request is proxied to Prometheus/VictoriaMetrics by [httputil](https://pkg.go.dev/net/http/httputil). The process is described in more details [here](docs/filtering.md).\n\nTarget setup: `grafana (with OIDC integration) \u003c-\u003e lfgw \u003c-\u003e Prometheus/VictoriaMetrics`.\n\n## Key features\n\n* ACL-based request rewrites with implicit deny;\n* a user can have multiple roles;\n* support for autoconfiguration in environments, where OIDC-role names match names of namespaces (\"assumed roles\" mode; thanks to [@aberestyak](https://github.com/aberestyak/) for the idea);\n* [automatic expression optimizations](https://pkg.go.dev/github.com/VictoriaMetrics/metricsql#Optimize) for non-full access requests;\n* support for different headers with access tokens (`Authorization`, `X-Forwarded-Access-Token`, `X-Auth-Request-Access-Token`), which can be useful for tools like [oauth2-proxy](https://github.com/oauth2-proxy/oauth2-proxy);\n* requests to both `/api/*` and `/federate` endpoints are protected (=rewritten);\n* requests to sensitive endpoints are blocked by default;\n* compatible with both [PromQL](https://prometheus.io/docs/prometheus/latest/querying/basics/) and [MetricsQL](https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/MetricsQL).\n\n## Similar projects\n\nSimilar projects are described [here](docs/similar-projects.md).\n\n## Docker images\n\nDocker images are published on [ghcr.io/weisdd/lfgw](https://github.com/weisdd/lfgw/pkgs/container/lfgw).\n\n## Configuration\n\nExample of `keycloak + grafana + lfgw` setup is described [here](docs/oidc.md).\n\n### Requirements for jwt-tokens\n\n* OIDC-roles must be present in `roles` claim;\n* Client ID specified via `OIDC_CLIENT_ID` must be present in `aud` claim (more details in [environment variables section](#environment-variables)), otherwise token verification will fail.\n\n### Environment variables\n\n#### Basic settings\n\n| Variable                    | Default Value | Description                                                  |\n| --------------------------- | ------------- | ------------------------------------------------------------ |\n| `UPSTREAM_URL`              |               | Prometheus URL, e.g. `http://prometheus.localhost`.          |\n| `OIDC_REALM_URL`            |               | OIDC Realm URL, e.g. `https://keycloak.localhost/auth/realms/monitoring` |\n| `OIDC_CLIENT_ID`            |               | OIDC Client ID (1*)                                          |\n| `ACL_PATH`                  | `./acl.yaml`  | Path to a file with ACL definitions (OIDC role to namespace bindings). Skipped if `ACL_PATH` is empty (might be useful when autoconfiguration is enabled through `ASSUMED_ROLES=true`). |\n| `ASSUMED_ROLES`             | `false`       | In environments, where OIDC-role names match names of namespaces, ACLs can be constructed on the fly (e.g. `[\"role1\", \"role2\"]` will give access to metrics from namespaces `role1` and `role2`). The roles specified in `acl.yaml` are still considered and get merged with assumed roles. Role names may contain regular expressions, including the admin definition `.*`. |\n\n(1*): since it's grafana who obtains jwt-tokens in the first place, the specified client id must also be present in the forwarded token (the `aud` claim).\n\n#### Additional settings\n\n| Variable                    | Default Value | Description                                                  |\n| --------------------------- | ------------- | ------------------------------------------------------------ |\n| `ENABLE_DEDUPLICATION`      | `true`        | Whether to enable deduplication, which leaves some of the requests unmodified if they match the target policy. Examples can be found in the \"acl.yaml syntax\" section. |\n| `OPTIMIZE_EXPRESSIONS`      | `true`        | Whether to automatically optimize expressions for non-full access requests. [More details](https://pkg.go.dev/github.com/VictoriaMetrics/metricsql#Optimize) |\n| `SAFE_MODE`                 | `true`        | Whether to block requests to sensitive endpoints like `/api/v1/admin/tsdb`, `/api/v1/insert`. |\n| `SET_PROXY_HEADERS`         | `false`       | Whether to set proxy headers (`X-Forwarded-For`, `X-Forwarded-Proto`, `X-Forwarded-Host`). |\n| `SET_GOMAXPROCS`            | `true`        | Automatically set `GOMAXPROCS` to match Linux container CPU quota. |\n| `DEBUG`                     | `false`       | Whether to print out debug log messages.                     |\n| `LOG_FORMAT`                | `pretty`      | Log format (`pretty`, `json`)                                |\n| `LOG_NO_COLOR`              | `false`       | Whether to disable colors for `pretty` format                |\n| `LOG_REQUESTS`              | `false`       | Whether to log HTTP requests                                 |\n| `PORT`                      | `8080`        | Port the web server will listen on.                          |\n| `READ_TIMEOUT`              | `10s`         | `ReadTimeout` covers the time from when the connection is accepted to when the request body is fully read (if you do read the body, otherwise to the end of the headers). [More details](https://blog.cloudflare.com/the-complete-guide-to-golang-net-http-timeouts/) |\n| `WRITE_TIMEOUT`             | `10s`         | `WriteTimeout` normally covers the time from the end of the request header read to the end of the response write (a.k.a. the lifetime of the ServeHTTP). [More details](https://blog.cloudflare.com/the-complete-guide-to-golang-net-http-timeouts/) |\n| `GRACEFUL_SHUTDOWN_TIMEOUT` | `20s`         | Maximum amount of time to wait for all connections to be closed. [More details](https://pkg.go.dev/net/http#Server.Shutdown) |\n\n### ACL syntax\n\nThe file with ACL definitions (`./acl.yaml` by default) has a simple structure:\n\n```yaml\nrole: namespace, namespace2\n```\n\nFor example:\n\n```yaml\nteam0: .*                # all metrics\nteam1: min.*, .*, stolon # all metrics, it's the same as .*\nteam2: minio             # only those with namespace=\"minio\"\nteam3: min.*             # only those matching namespace=~\"min.*\"\nteam4: minio, stolon     # only those matching namespace=~\"minio|stolon\"\nteam5: min.*, stolon     # only those matching namespace=~\"min.*|stolon\"\n```\n\nTo summarize, here are the key principles used for rewriting requests:\n\n* `.*` - all requests are simply forwarded to an upstream;\n* `minio` - all label filters with the `namespace` label are removed, then `namespace=\"minio\"` is added;\n* `min.*` -  positive regex-match label filters (`namespace=~\"X\"`) are removed, then `namespace=~\"mi.*\"` is added;\n* `minio, stolon` - positive regex-match label filters (`namespace=~\"X\"`) are removed, then `namespace=~\"minio|stolon\"` is added;\n* `min.*, stolon` - positive regex-match label filters (`namespace=~\"X\"`) are removed, then `namespace=~\"min.*|stolon\"` is added.\n\nWhen deduplication is enabled, these queries will stay unmodified:\n\n* `min.*, stolon`, query: `request_duration{namespace=\"minio\"}` - a non-regexp label filter that matches policy;\n* `min.*, stolon`, query: `request_duration{namespace=~\"minio\"}` - a \"fake\" regexp (no special symbols) label filter that matches policy;\n* `min.*, stolon`, query: `request_duration{namespace=~\"min.*\"}` - a label filter is a subfilter of the policy.\n\nNote: Regex matches are fully anchored. A match of `env=~\"foo\"` is treated as `env=~\"^foo$\"` ([Source](https://prometheus.io/docs/prometheus/latest/querying/basics/)). Please, be careful, they are not expected to be used in ACLs.\n\nNote: a user is free to have multiple roles matching the contents of `acl.yaml`. Basically, there are 3 cases:\n\n* one role\n  =\u003e a prepopulated LF is returned;\n* multiple roles, one of which gives full access\n  =\u003e a prepopulated LF, corresponding to the full access role, is returned;\n* multiple \"limited\" roles\n  =\u003e definitions of all those roles are merged together, and then lfgw generates a new LF. The process is the same as if this meta-definition was loaded through `acl.yaml`.\n\n## Licensing\n\nlfgw code is licensed under MIT, though its dependencies might have other licenses. Please, inspect the modules listed in [go.mod](go.mod) if needed.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fweisdd%2Flfgw","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fweisdd%2Flfgw","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fweisdd%2Flfgw/lists"}