{"id":34599168,"url":"https://github.com/flyingrobots/git-kv","last_synced_at":"2026-03-15T01:38:57.678Z","repository":{"id":319760701,"uuid":"1079469469","full_name":"flyingrobots/git-kv","owner":"flyingrobots","description":"State as code. Commits are transactions. Git is the database.","archived":false,"fork":false,"pushed_at":"2025-10-20T04:02:18.000Z","size":168,"stargazers_count":0,"open_issues_count":4,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-20T05:48:21.955Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Makefile","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/flyingrobots.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-19T21:35:16.000Z","updated_at":"2025-10-20T00:29:41.000Z","dependencies_parsed_at":"2025-11-10T09:45:17.387Z","dependency_job_id":null,"html_url":"https://github.com/flyingrobots/git-kv","commit_stats":null,"previous_names":["flyingrobots/git-kv"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/flyingrobots/git-kv","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flyingrobots%2Fgit-kv","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flyingrobots%2Fgit-kv/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flyingrobots%2Fgit-kv/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flyingrobots%2Fgit-kv/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/flyingrobots","download_url":"https://codeload.github.com/flyingrobots/git-kv/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flyingrobots%2Fgit-kv/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28002250,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-12-24T02:00:07.193Z","response_time":83,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-12-24T12:07:35.764Z","updated_at":"2025-12-24T12:07:49.499Z","avatar_url":"https://github.com/flyingrobots.png","language":"Makefile","funding_links":[],"categories":[],"sub_categories":[],"readme":"# git-kv (Project Stargate)\n\n**State as code. Commits are transactions. Git is the database. GitHub is the showroom.**\n\n\u003e [!warning]\n\u003e `git-kv` is just getting started! The [SPEC](./docs/SPEC.md) just dropped, and we are about to break ground on the implementation. \n\u003e \n\u003e This `README` is basesd on the spec, and is mostly here to generate hype and interest.   \n\u003e None of this is true (yet), but watch the repo to follow along.   \n\u003e Expect things to change as we build.\n\u003e \n\u003e **COMING SOON:** GitHub Issues to track outstanding tasks, organized into **Milestones** according to the roadmap, which you can find in the spec doc.\n\n`git-kv` is an auditable, versioned Key-Value store that uses Git *as* the database. It provides ledger-grade integrity, atomic multi-key transactions, and fast prefix-listing, all while remaining compatible with SaaS Git hosts like GitHub and GitLab.\n\nIt achieves this with a transparent \"Stargate\" gateway:\n\n* **Reads (`git fetch`)** pull from a GitHub mirror for CDN-class speed.\n* **Writes (`git push`)** are routed to a tiny, self-hosted Stargate gateway that enforces all integrity guarantees before mirroring back to GitHub.\n\n---\n\n## Why git-kv?\n\n`git-kv` is designed for workflows where **audit beats microseconds**. It's a durable ledger for config, policies, and state, **not a Redis replacement** for 100k QPS.\n\n* **Ledger-Grade History:** Every change is a signed, immutable, linear-history Git commit. No merge commits, no force-pushes.\n* **Atomic Multi-Key Transactions:** Set 10,000 keys and delete 5,000 in a single, atomic transaction (`mset`). This is validated on the server in **O(1)** time via client attestation.\n* **Fast Prefix Listing:** A dedicated `kv-index` ref provides fast `git kv list --prefix user:` operations without scanning all N keys.\n* **Native Large File Support:** Automatically chunks values \\\u003e1MB using Content-Defined Chunking (FastCDC). This provides high-performance deduplication *inside* Git, with no Git-LFS required.\n* **Bounded Clone Size:** **Epochs** (`git kv epoch new`) snapshot the database, keeping new clones small and fast.\n* **SaaS-Native Ergonomics:** Developers work against `origin` (GitHub/GitLab) as usual. The Stargate is a transparent proxy on the `push` path, giving you server-side hooks and guarantees without sacrificing your existing UI/CI/Actions workflow.\n\n---\n\n## Core Concepts: The \"Stargate\" Architecture\n\n`git-kv` works by separating the read and write paths.\n\n```bash\ndev clone\n  git fetch origin  ─────────→  GitHub (CDN, UI, Read Path)\n  git push origin   ─────────→  Stargate (LAN, Authoritative Write Path)\n                                   │ pre-receive (O(1) attestation, policy)\n                                   └─ post-receive mirror → GitHub\n```\n\nThis transparency is achieved with a simple Git config rule, set by `git kv remote setup`:\n\n```ini\n[remote \"origin\"]\n  url     = git@github.com:your-org/repo.git         # Reads\n  pushurl = ssh://git@stargate.local/org/repo.git    # Writes\n```\n\nThe **Stargate** is a minimal, self-hosted daemon that runs your bare repo and enforces all guarantees. It stores data across three core ref types, which are pushed atomically:\n\n1.  `refs/kv/\u003cns\u003e` (**Ledger**): The authoritative, hash-sharded data (`data/`) and metadata (`meta/`).\n2.  `refs/kv-index/\u003cns\u003e` (**Index**): Prefix-sharded pointer files (`.ref`) for fast listing.\n3.  `refs/kv-chunks/\u003cns\u003e@\u003cepoch\u003e` (**Chunks**): Content-addressed blobs for large values.\n\n---\n\n## Quick Start\n\n### 1\\. Admin: Set up Stargate\n\n(One-time setup)\n\n1.  Deploy the `git-kv-stargate` binary to a server (`stargate.local`).\n2.  Initialize it as a bare repo with hooks and policy:\n    ```bash\n    # On the Stargate server\n    git-kv-stargate init --repo /srv/git/kv.git --mirror-remote git@github.com:your-org/kv.git\n    systemctl start git-kv-stargate\n    ```\n3.  On GitHub, configure the repo's `refs/kv/*` branches as \"protected\" to block all direct pushes.\n\n### 2\\. Developer: Clone \u0026 Bootstrap\n\n(Each developer, one time)\n\n1.  Install the `git-kv` client CLI.\n    ```bash\n    # (Installation instructions TBD: e.g., go install ... or download from releases)\n    ```\n2.  Clone the repo from GitHub as usual:\n    ```bash\n    git clone git@github.com:your-org/kv.git\n    cd kv\n    ```\n3.  Run the `remote setup` command. This reads `.kv/policy.yaml` from the repo and configures your `pushurl` automatically.\n    ```bash\n    git kv remote setup\n    # \u003e Configured origin.pushurl → ssh://git@stargate.local/org/kv.git\n    ```\n\n### 3\\. Developer: Read \u0026 Write\n\n```bash\n# Set a simple value (this creates one atomic commit)\ngit kv set cfg:feature:new-ui \"true\"\n\n# Get the value\ngit kv get cfg:feature:new-ui\n# \u003e \"true\"\n\n# Perform an atomic multi-key transaction\ngit kv mset --file changes.jsonl\n\n# changes.jsonl\n# {\"op\":\"set\",\"key\":\"cfg:feature:new-ui\",\"ctype\":\"application/json\",\"value_b64\":\"eyJhY3RpdmUiOnRydWV9\"}\n# {\"op\":\"set\",\"key\":\"user:42:avatar\",\"ctype\":\"image/png\",\"file\":\"avatar.png\"}\n# {\"op\":\"del\",\"key\":\"secret:old-token\"}\n\n# List all keys under a prefix (fast!)\ngit kv list --prefix cfg:\n# \u003e cfg:feature:new-ui\n\n# In CI, wait for a change to be visible on the GitHub mirror\ngit kv wait --oid \u003ccommit-hash\u003e --visible-on=github\n```\n\n---\n\n## Core Commands\n\n### Developer CLI\n\n* `git kv remote setup`: **(First command)** Configures `origin.pushurl` to point to Stargate.\n* `git kv get \u003ckey\u003e`: Reads a value.\n* `git kv set \u003ckey\u003e \u003cvalue\u003e`: Sets a value (string, file, or stdin).\n* `git kv del \u003ckey\u003e`: Deletes a value (writes a tombstone).\n* `git kv mset --file \u003cbatch.jsonl\u003e`: Executes a list of `set`/`del` ops atomically.\n* `git kv list [--prefix P]`: Lists keys (fast, uses the index).\n* `git kv history \u003ckey\u003e`: Shows the commit log for a single key.\n* `git kv wait --oid \u003chash\u003e`: **(For CI)** Pauses a script until a commit is visible on the GitHub mirror.\n\n### Admin (Stargate) CLI\n\n* `git kv stargate status`: Checks the health and mirror backlog of the Stargate.\n* `git kv stargate sync [--repair]`: Audits and repairs a diverged GitHub mirror.\n* `git kv epoch new --label \u003cname\u003e`: Creates a new, bounded snapshot for fast clones.\n\n-----\n\n## Advanced Features\n\n* **Large Value Chunking**: Automatic for files \\\u003e `max_value_inline` (e.g., 1MB). Uses FastCDC to minimize delta size.\n* **Read-Side TTL**: Set a `ttl` in the metadata; `git kv get` will return \"not found\" (exit 7) if it's expired.\n* **Policy Enforcement**: The `.kv/policy.yaml` file (enforced by Stargate) controls:\n  * `stargate.push_url`\n  * Allowed signer keys (SSH/GPG)\n  * Allowed key prefixes (`user:*`, `cfg:*`)\n  * Max value sizes\n\n---\n\n## Stargate Operations (For SREs)\n\nThe Stargate gateway is a critical service that must be run with **High Availability** for production use.\n\n* **HA Model**: **Active-Passive** is the recommended deployment. A floating IP (`stargate.local`) points to the leader, which holds a leader lock. The standby continuously mirrors the primary.\n* **Health Checks**:\n  * `/healthz`: Is the repo readable and are hooks loaded?\n  * `/readyz`: Is this node the leader and is the mirror remote reachable?\n* **Observability**: Stargate exposes a `/metrics` endpoint for Prometheus with key SLOs:\n  * `stargate_txn_latency_ms` (p95 \\\u003c 150ms)\n  * `stargate_mirror_lag_seconds` (p95 \\\u003c 2s)\n  * `stargate_mirror_backlog_size`\n* **Recovery**: If the GitHub mirror *ever* diverges from the Stargate (Source of Truth), an admin can force a repair:\n    ```bash\n    git kv stargate sync --repair --ns main\n    ```\n\n---\n\n## License\n\nThis project is licensed under the **[MIND-UCAL License](https://github.com/universalcharter/mind-ucal)**. See `LICENSE` file for details.\n\n## Contributing\n\nPull Requests are welcome for issues marked with the `Help Wanted` label\\! \n\n*Please* run `make test` and `make lint` before submitting. \n\n**For major architectural changes**, please use Discussions to submit a RFC.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflyingrobots%2Fgit-kv","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fflyingrobots%2Fgit-kv","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflyingrobots%2Fgit-kv/lists"}