{"id":18310104,"url":"https://github.com/kazhuravlev/database-gateway","last_synced_at":"2026-04-01T17:49:23.253Z","repository":{"id":261055707,"uuid":"880483725","full_name":"kazhuravlev/database-gateway","owner":"kazhuravlev","description":"Safe access to production databases","archived":false,"fork":false,"pushed_at":"2026-03-19T03:12:54.000Z","size":6361,"stargazers_count":33,"open_issues_count":10,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-03-29T16:58:01.768Z","etag":null,"topics":["acl","database","database-access","database-security","devsecops","gateway","postgres","production","security"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kazhuravlev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-10-29T20:05:47.000Z","updated_at":"2026-03-19T14:02:04.000Z","dependencies_parsed_at":"2026-03-06T23:07:29.717Z","dependency_job_id":null,"html_url":"https://github.com/kazhuravlev/database-gateway","commit_stats":null,"previous_names":["kazhuravlev/database-gateway"],"tags_count":92,"template":false,"template_full_name":null,"purl":"pkg:github/kazhuravlev/database-gateway","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kazhuravlev%2Fdatabase-gateway","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kazhuravlev%2Fdatabase-gateway/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kazhuravlev%2Fdatabase-gateway/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kazhuravlev%2Fdatabase-gateway/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kazhuravlev","download_url":"https://codeload.github.com/kazhuravlev/database-gateway/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kazhuravlev%2Fdatabase-gateway/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31290621,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-01T13:12:26.723Z","status":"ssl_error","status_checked_at":"2026-04-01T13:12:25.102Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["acl","database","database-access","database-security","devsecops","gateway","postgres","production","security"],"created_at":"2024-11-05T16:13:15.358Z","updated_at":"2026-04-01T17:49:23.244Z","avatar_url":"https://github.com/kazhuravlev.png","language":"Go","funding_links":[],"categories":["Database","数据库","Data Integration Frameworks"],"sub_categories":["Database Tools","数据库工具"],"readme":"# Database Gateway\n\n[![Go Reference](https://pkg.go.dev/badge/github.com/kazhuravlev/database-gateway.svg)](https://pkg.go.dev/github.com/kazhuravlev/database-gateway)\n[![License](https://img.shields.io/github/license/kazhuravlev/database-gateway?color=blue)](https://github.com/kazhuravlev/database-gateway/blob/master/LICENSE)\n[![Test Status](https://github.com/kazhuravlev/database-gateway/actions/workflows/test.yml/badge.svg)](https://github.com/kazhuravlev/database-gateway/actions/workflows/test.yml)\n[![Release Status](https://github.com/kazhuravlev/database-gateway/actions/workflows/release.yml/badge.svg)](https://github.com/kazhuravlev/database-gateway/actions/workflows/release.yml)\n[![Go Report Card](https://goreportcard.com/badge/github.com/kazhuravlev/database-gateway)](https://goreportcard.com/report/github.com/kazhuravlev/database-gateway)\n[![codecov](https://codecov.io/gh/kazhuravlev/database-gateway/graph/badge.svg?token=DLOML3FTN1)](https://codecov.io/gh/kazhuravlev/database-gateway)\n[![Mentioned in Awesome Go](https://awesome.re/mentioned-badge.svg)](https://github.com/avelino/awesome-go#database-tools)\n\nThis service provides a unified web interface for secure, controlled access to company databases. It enables employees\nto run queries on `production` databases while enforcing Open Policy Agent (`OPA`) policies. For example, team leads may\nhave permissions to execute both `SELECT` and `INSERT` queries on certain tables, while other team members are\nrestricted to read-only (`SELECT`) access. This approach ensures that database interactions are managed safely and\nthat each user's access is tailored to their role and responsibilities.\n\n## TL;DR\n\n- Run approved SQL against multiple PostgreSQL targets from one web UI.\n- Authenticate users via OIDC and enforce OPA rules by user, target, operation, and table.\n- Store query results (with shareable links and execution metadata) for debugging and auditing.\n\n## Table of Contents\n\n- [TL;DR](#tldr)\n- [Architecture Overview](#architecture-overview)\n- [Quickstart with example setup](#quickstart-with-example-setup)\n- [Features](#features)\n- [Advanced Configuration](#advanced-configuration)\n- [Performance Optimizations](#performance-optimizations)\n- [Security Considerations](#security-considerations)\n- [Edge Cases and Troubleshooting](#edge-cases-and-troubleshooting)\n- [Interesting projects](#interesting-projects)\n\n## Architecture Overview\n\nThis application acts as a secure gateway to multiple PostgreSQL instances, allowing authenticated users to run approved\nqueries through a unified web interface, with fine-grained OPA policies controlling access.\n\n\n```\n                     ┌───────────────────────────┐\n                     │ PROD      ┌─────────────┐ │\n                     │       ┌───┤  Postgres1  │ │\n  ┌────────┐    ┌────────┐   │   └─────────────┘ │\n  │  USER  │────│  DBGW  │───┼                   │\n  └────────┘    └────────┘   │   ┌─────────────┐ │\n                     │       └───┤  Postgres2  │ │\n                     │           └─────────────┘ │\n                     └───────────────────────────┘\n```\n\n### Components\n\n1. **Local PostgreSQL Database**:\n    - Stores query results and user profiles.\n    - Acts as a cache for query results, allowing unique links for debugging without re-execution.\n\n2. **Remote PostgreSQL Instances**:\n    - Host production data and are accessed only through the app.\n    - Queries are run only if authorized by OPA policies, limiting access to specific users, tables, and query types.\n\n3. **OIDC Authentication**:\n    - Users authenticate via an external OIDC provider.\n    - User roles are mapped to OPA subjects, defining what queries each user can run.\n\n4. **OPA Policies**:\n    - Define user permissions at the instance, table, and query type levels.\n    - Loaded from `.rego` files on disk and evaluated inside the gateway.\n\n5. **Web Interface**:\n    - Provides login, query submission, and result viewing.\n    - Shows error feedback for unauthorized or restricted queries.\n\n### Flow of Operations\n\n1. **Authentication**: Users log in via OIDC, and their identity maps to OPA subjects.\n2. **Query Submission**: Authorized queries are checked against OPA policies, then run on remote instances.\n3. **Result Caching**: Results are stored locally with unique links for easy access and debugging.\n\nThis architecture ensures secure, controlled access to production data, balancing usability with data protection.\n\n## Quickstart with example setup\n\nRun commands to get a local dbgw instance with 3 PostgreSQL instances.\n\n```shell\ngit clone https://github.com/kazhuravlev/database-gateway.git\ncd database-gateway/example\ndocker compose up --pull always --force-recreate -d\nopen 'http://localhost:8080'\n# Authentik and test users are bootstrapped automatically.\n```\n\nThe example setup uses self-hosted Authentik as the OIDC provider.\nOPA policies are loaded from `example/opa/basic/` and configured in [config.json](example/config.json).\nUse `localhost` consistently for the example login flow, because the example OIDC redirect URL is\n`http://localhost:8080/auth/callback`.\n\nBootstrap details for local Authentik:\n\n1. OIDC app is created from `example/authentik-blueprint.yaml`.\n2. Static users are created with password `password`:\n   - `admin@example.com`\n   - `user1@example.com`\n3. `admin@example.com` belongs to `dbgw-admins` and `dbgw-users`; `user1@example.com` belongs to `dbgw-users`.\n4. Authentik admin user:\n   - `akadmin@example.com` / `password`\n\n![pic1_instances.png](example/list_instances.png)\n\nChoose `local-1`, run this query `select id, name from clients`, then click `Run`. ![pic2_run.png](example/instance.png)\n\nAdmins can inspect recent stored requests and open a detailed result view with execution metadata and exported formats.\n\n![admin-query-list.png](example/admin-query-list.png)\n\n![admin-query-inspect.png](example/admin-query-inspect.png)\n\n## Features\n\n### Security \u0026 Access Control\n\n- [x] Integrates with OpenID Connect for user authentication\n- [x] Enforces access filtering through OPA\n- [x] Fine-grained table-level permissions\n- [x] Schema-backed column allowlists\n- [x] SQL parsing to enforce query type restrictions (SELECT, INSERT, etc.)\n- [x] Query validation and sanitization\n- [x] Session management with token expiration\n- [x] Secure cookie handling\n\n### Query UX\n\n- [x] Supports any PostgreSQL wire-protocol database\n- [x] Interactive web UI with keyboard shortcuts (Shift+Enter to run queries)\n- [x] Provides query result output in HTML format\n- [x] Provides query result output in JSON format\n- [x] Query bookmarks (save, list, run, delete)\n- [x] Recent queries feed on the main page (last 50 per user) with quick result access\n- [x] Unique links for query results (useful for debugging)\n\n### LRPC API\n\nLRPC endpoint is exposed on the same port as the web facade:\n\n- `GET /api/v1/token` (returns current session access token for frontend app)\n- `POST /api/v1/:method`\n- `GET /api/v1/schema`\n- `GET /api/v1/query-results/export/:token`\n\nAvailable methods:\n\n- `targets.list.v1` - list user-available targets\n- `targets.get.v1` - get a single target by `target_id`\n- `bookmarks.list.v1` - list all bookmarks, or filter by optional `target_id`\n- `bookmarks.add.v1` - save a bookmark for `target_id`, `title`, and `query`\n- `bookmarks.delete.v1` - delete a bookmark by `id`\n- `queries.list.v1` - list recent queries, with optional `limit`\n- `query.run.v1` - run query for a target and return table data\n- `query-results.get.v1` - get stored query result by `query_result_id`; users can read their own results and admins can read any user's result\n- `query-results.export-link.v1` - issue a short-lived export link for `json` or `csv`\n\nDownload endpoints:\n\n- `/api/v1/query-results/export/:token` - download an exported file using a short-lived signed token\n\nAll API requests require an OIDC access token in the header:\n\n```json\nAuthorization: Bearer \u003caccess_token\u003e\n```\n\n`params` are method-specific. User identity and role are resolved from the verified token claims.\n\n### Observability \u0026 Performance\n\n- [x] Includes query execution stats in results (full round trip, parsing time, network round trip)\n- [x] Connection pooling for performance optimization\n\n## Advanced Configuration\n\n### Authentication\n\nThe service uses OIDC authentication:\n\n```json\n{\n  \"users\": {\n    \"client_id\": \"db-gateway\",\n    \"client_secret\": \"db-gateway-secret\",\n    \"issuer_url\": \"http://localhost:9000/application/o/db-gateway/\",\n    \"redirect_url\": \"http://localhost:8080/auth/callback\",\n    \"access_token_audience\": \"db-gateway\",\n    \"scopes\": [\"groups\", \"email\", \"profile\"],\n    \"role_claim\": \"groups\",\n    \"role_mapping\": {\n      \"dbgw-admins\": \"admin\",\n      \"dbgw-users\": \"user\"\n    }\n  }\n}\n```\n\n`access_token_audience` is optional. If omitted, `client_id` is used for access-token audience validation.\n\n### Policy Configuration\n\nOPA policy bundles are loaded from disk:\n\n```json\n{\n  \"policy\": {\n    \"path\": \"./opa/basic\"\n  }\n}\n```\n\nEach `.rego` file in the configured directory is compiled into the embedded OPA authorizer. Policies must define:\n\n- `data.gateway.allow_target`\n- `data.gateway.allow_query`\n\n`policy.path` is resolved relative to the config file when it is not absolute.\n\nCurrent OPA input:\n\n```json\n{\n  \"subjects\": [\"user:alice@example.com\", \"role:user\"],\n  \"target\": \"local-1\",\n  \"op\": \"select\",\n  \"table\": \"public.clients\"\n}\n```\n\nNotes:\n\n- `subjects` always includes both the concrete user principal and the mapped role principal\n- `table` is always sent to OPA in canonical `schema.table` form\n- unqualified SQL like `select id from clients` is normalized before policy evaluation\n- policies run once for target visibility and once for each parsed query vector\n\n### Database Connection Settings\n\nConfigure performance settings for each database connection:\n\n```json\n{\n  \"connection\": {\n    \"host\": \"postgres1\",\n    \"port\": 5432,\n    \"user\": \"pg01\",\n    \"password\": \"pg01\",\n    \"db\": \"pg01\",\n    \"use_ssl\": false,\n    \"max_pool_size\": 4\n  }\n}\n```\n\nFor a complete working config, see [example/config.json](example/config.json).\n\n## Performance Optimizations\n\n- **Connection Pooling**: Configurable connection pool sizes for each database target\n- **Query Result Caching**: Results are stored in the local database for later reference\n- **Efficient Query Execution**: Parsed and validated for optimal performance\n\n## Security Considerations\n\n- **SQL Injection Protection**: All queries are parsed and validated before execution\n- **No Direct Database Access**: Remote databases are only accessible through the gateway\n- **Column-Level Restrictions**: schema validation limits which fields users can query\n- **Query Type Restrictions**: Limit users to specific operations (SELECT, INSERT, etc.)\n- **Session Security**: Secure cookie handling with configurable expiration\n- **Error Handling**: Error messages are sanitized to prevent information leakage\n\n## Edge Cases and Troubleshooting\n\n- **Multiple Schema Support**: Tables can be specified with schema names (`schema.table`)\n- **Complex Query Handling**: Some complex queries might be rejected by the parser\n- **Connection Failures**: The service gracefully handles database connection failures\n- **Missing Tables/Fields**: Queries referencing unknown tables or fields are rejected\n- **Policy Compilation Errors**: invalid `.rego` files fail startup\n\n## Interesting projects\n\n- https://github.com/antlr/grammars-v4/tree/master/sql/postgresql/Go\n- https://github.com/auxten/postgresql-parser\n- https://github.com/blastrain/vitess-sqlparser\n- https://github.com/cockroachdb/cockroach/pkg/sql/parser\n- https://github.com/pganalyze/pg_query_go/\n- https://github.com/pingcap/tidb/tree/master/pkg/parser\n- https://github.com/topics/sql-parser?l=go\n- https://github.com/vitessio/vitess\n- https://github.com/xwb1989/sqlparser\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkazhuravlev%2Fdatabase-gateway","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkazhuravlev%2Fdatabase-gateway","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkazhuravlev%2Fdatabase-gateway/lists"}