{"id":47247904,"url":"https://github.com/franchoy/coldkeep","last_synced_at":"2026-04-09T09:01:27.953Z","repository":{"id":341998287,"uuid":"1158711822","full_name":"franchoy/coldkeep","owner":"franchoy","description":"coldkeep is an experimental local-first content-addressed file storage engine with verifiable integrity written in Go. Files are split into content-addressed chunks, packed into container files on disk, and tracked through PostgreSQL metadata to attempt deduplication","archived":false,"fork":false,"pushed_at":"2026-03-29T20:01:08.000Z","size":14659,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-03-29T20:23:53.853Z","etag":null,"topics":["backup","cold-storage","content-defined-chunking","deduplicate","go","research-project","storage-engines"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/franchoy.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-15T19:58:15.000Z","updated_at":"2026-03-28T10:29:59.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/franchoy/coldkeep","commit_stats":null,"previous_names":["franchoy/coldkeep"],"tags_count":8,"template":false,"template_full_name":null,"purl":"pkg:github/franchoy/coldkeep","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/franchoy%2Fcoldkeep","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/franchoy%2Fcoldkeep/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/franchoy%2Fcoldkeep/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/franchoy%2Fcoldkeep/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/franchoy","download_url":"https://codeload.github.com/franchoy/coldkeep/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/franchoy%2Fcoldkeep/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31290943,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-01T13:12:26.723Z","status":"ssl_error","status_checked_at":"2026-04-01T13:12:25.102Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backup","cold-storage","content-defined-chunking","deduplicate","go","research-project","storage-engines"],"created_at":"2026-03-14T09:27:27.024Z","updated_at":"2026-04-09T09:01:27.938Z","avatar_url":"https://github.com/franchoy.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/logo/coldkeep-logo.png\" alt=\"Coldkeep Logo\" width=\"500\"/\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  Correctness-first cold storage engine\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  • Content-addressed • Built-in deduplication • Deterministic restore • Verifiable integrity • Crash-safe • GC-safe\n\u003c/p\u003e\n\n## 🧊 Branding\n\n\u003cp align=\"left\"\u003e\n  \u003cimg src=\"assets/logo/coldkeep-icon.png\" alt=\"Coldkeep Icon\" width=\"120\"/\u003e\n\u003c/p\u003e\n\nColdkeep uses a visual identity based on an ice cube vault:\n\n- cold storage (ice cube)\n- secure data (vault door)\n- structured containers (internal shelves)\n\n# coldkeep\n\n![CI](https://github.com/franchoy/coldkeep/actions/workflows/ci.yml/badge.svg)\n![Go Version](https://img.shields.io/badge/go-1.23+-blue)\n![License](https://img.shields.io/badge/license-Apache%202.0-blue)\n![Status](https://img.shields.io/badge/status-v1.1%20interface--correctness%20milestone-brightgreen)\n![Release](https://img.shields.io/github/v/release/franchoy/coldkeep?include_prereleases)\n\n\u003e Status: v1.1 adds an interface-correctness layer on top of the v1.0 storage-correctness core.\n\ncoldkeep is a local-first content-addressed storage engine focused on deterministic restore,\nexplicit integrity verification, and safe lifecycle behavior under failure scenarios.\n\n## Why coldkeep?\n\ncoldkeep is designed for correctness-first cold storage.\n\nUnlike traditional backup tools, it emphasizes:\n\n- deterministic, byte-identical restore\n- content-addressed deduplication\n- explicit, test-backed integrity checks\n- safe recovery and reference-safe garbage collection\n- machine-readable CLI behavior suitable for automation\n\nThe goal is confidence and recoverability over maximum throughput.\n\n## Status\n\nColdkeep currently has two explicit correctness layers:\n\n- v1.0: storage correctness (restore determinism, integrity, recovery, GC safety)\n- v1.1: interaction correctness (CLI orchestration, machine-readable contracts, batch semantics)\n\nGuarantees are enforced through automated validation and CI gates.\nSee VALIDATION_MATRIX.md for guarantee-to-evidence mapping.\n\n## Core Guarantees\n\n### Summary\n\n- deterministic, byte-identical restore\n- no exposure of partially written or inconsistent data\n- GC is reference-safe: no reachable chunk is ever deleted\n- Atomic restore replacement (within single-node local filesystem semantics)\n- Safe in-process concurrent storage operations\n\n### Core invariants\n\nGuarantee IDs are stable and tracked in VALIDATION_MATRIX.md:\n\n- G1: deterministic, byte-identical restore\n- G2: repeat store does not drift chunk graph\n- G3: no exposure of partially written or inconsistent data\n- G4: GC is reference-safe (no reachable chunk is deleted)\n- G5: atomic restore replacement (single-node local filesystem semantics)\n- G6: safe in-process concurrent storage operations\n- G7: deep corruption detection (payload/offset/tail)\n- G8: corrective health gate contract stability\n- G9: deterministic batch CLI orchestration and automation-safe contract behavior\n\nColdkeep separates system understanding into:\n\n- README.md (overview and usage)\n- ARCHITECTURE.md (internal model and invariants)\n\nFor the deep model (invariants, lifecycle, validity, recovery, trust boundary), see ARCHITECTURE.md.\n\n## When to use coldkeep\n\nGood fit:\n\n- cold/backup storage where correctness matters more than speed\n- environments needing explicit integrity verification\n- deduplication + deterministic restore use cases\n\nNot a fit (v1.x scope):\n\n- hot-path high-throughput storage\n- distributed/multi-node coordination\n\n## Quickstart\n\nA small samples directory is included for local testing.\n\n### Local (no Docker)\n\n```bash\n# 1) Initialize key material (.env)\ncoldkeep init\n\n# 2) Load environment\nexport $(cat .env | xargs)\n\n# 3) Store and inspect\ncoldkeep store samples/hello.txt\ncoldkeep stats\n\n# 4) Restore\ncoldkeep restore 1 ./restored\n```\n\nSecurity note: if the encryption key is lost, encrypted data cannot be recovered.\n\n### Docker\n\n```bash\n# 1) Start services\ndocker compose up -d --build\n\n# 2) Initialize key material on host-mounted workspace\ndocker compose run --rm -v \"$PWD:/app\" coldkeep init\n\n# 3) Store a sample file\ndocker compose run --rm \\\n  --env-file .env \\\n  -v \"$PWD/samples:/samples\" \\\n  coldkeep store /samples/hello.txt\n```\n\n## CLI Basics\n\nTypical flows:\n\n```bash\ncoldkeep store file.txt\ncoldkeep store-folder ./data\ncoldkeep restore 12 ./out\ncoldkeep remove 12\ncoldkeep gc\ncoldkeep stats\ncoldkeep list\ncoldkeep search report\ncoldkeep verify system --standard\ncoldkeep doctor\n```\n\nSimulation (no physical writes):\n\n```bash\ncoldkeep simulate store-folder ./data\ncoldkeep simulate store file.txt --output json\n```\n\n## Batch Operations (v1.1)\n\nBatch restore/remove expands v1.1 interface correctness for automation.\n\n```bash\ncoldkeep restore 12 18 24 ./out\ncoldkeep remove 12 18 24\ncoldkeep remove --input ids.txt\ncoldkeep restore 12 18 ./out --dry-run\n```\n\nSemantics (summary):\n\n- per-item isolation by default\n- optional fail-fast for execution failures\n- duplicate target skipping\n- deterministic per-item report ordering\n- JSON status values: ok, partial_failure, error\n- process exit is automation-friendly:\n  - 0 when no item fails\n  - 1 when one or more items fail\n\nClarifier: the binary 0/1 mapping applies to executed batch reports. Pre-execution validation/usage failures (including empty effective target sets after parsing input) return usage exit code 2.\n\nFor full batch contract details and examples, see ARCHITECTURE.md and PRE_RELEASE_CHECKLIST.md.\n\n## Doctor (recommended health gate)\n\ncoldkeep doctor is the operator health gate:\n\n- runs recovery first (corrective)\n- then schema/version sanity checks\n- then verification (standard by default; full/deep optional)\n\nDoctor is intentionally corrective, not read-only.\n\n```bash\ncoldkeep doctor\ncoldkeep doctor --full\ncoldkeep doctor --deep --output json\n```\n\n## Verification\n\nVerification levels:\n\n- standard: metadata integrity\n- full: structural/container integrity\n- deep: full content read + hash validation\n\n```bash\ncoldkeep verify system --standard\ncoldkeep verify system --full\ncoldkeep verify system --deep\n```\n\nVerification checks are observational. In CLI flows, startup recovery may run before verification.\n\n## Documentation Map\n\n- Architecture and internals: ARCHITECTURE.md\n- Guarantee mapping and evidence: VALIDATION_MATRIX.md\n- Contribution workflow: CONTRIBUTING.md\n- Release readiness flow: PRE_RELEASE_CHECKLIST.md\n- Security reporting and threat guidance: SECURITY.md\n\n## Roadmap note (v1.2 and beyond)\n\nv1.2 is planned to introduce a physical_file to logical_file mapping layer to preserve\nfilesystem structure semantics more explicitly. This is an extension of the architecture,\nnot a reset of the core correctness model (G1-G9).\n\n## Contributing\n\nContributions and discussions are welcome.\nSee CONTRIBUTING.md.\n\n## License\n\nApache-2.0. See LICENSE.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffranchoy%2Fcoldkeep","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffranchoy%2Fcoldkeep","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffranchoy%2Fcoldkeep/lists"}