{"id":26977190,"url":"https://github.com/phrozen/bitcask","last_synced_at":"2025-04-03T12:18:36.855Z","repository":{"id":146044265,"uuid":"473692043","full_name":"phrozen/bitcask","owner":"phrozen","description":"A pure Go implementation of the Bitcask model with zero dependencies","archived":false,"fork":false,"pushed_at":"2022-03-26T03:59:47.000Z","size":12,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-06-21T17:01:45.686Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/phrozen.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-03-24T16:43:31.000Z","updated_at":"2023-04-06T21:22:03.000Z","dependencies_parsed_at":null,"dependency_job_id":"eee37f48-b834-4299-a5a8-ee7563071665","html_url":"https://github.com/phrozen/bitcask","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/phrozen%2Fbitcask","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/phrozen%2Fbitcask/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/phrozen%2Fbitcask/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/phrozen%2Fbitcask/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/phrozen","download_url":"https://codeload.github.com/phrozen/bitcask/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246998187,"owners_count":20866696,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-04-03T12:18:35.642Z","updated_at":"2025-04-03T12:18:36.829Z","avatar_url":"https://github.com/phrozen.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# bitcask\n\n**Work In Progress**\n\nA pure Go implementation of the [BitCask](https://riak.com/assets/bitcask-intro.pdf) model with no dependecies (hopefully).\n\nIt is NOT meant to be a 1:1 replacement for Bitcask as [Riak](https://riak.com/) uses it, nor it is meant to be fully compatible with original Erlang's Bitcask database files.\n\nIt IS meant to be an idiomatic Go implementation of the Log-Structured Merge (LSM) Trees with a Write Ahead Log (WAL) model that Bitcask uses. It IS meant to be performant and easy to read through the code, and will implement AT LEAST all the basic features described on [BitCask's model design paper](https://riak.com/assets/bitcask-intro.pdf).\n\n## Introduction\n\n\u003e Bitcask is an Erlang application that provides an API for storing and retrieving key/value data into a log-structured hash table.\n\nIt is a fast persistent key/value store with predictable performance and strong fault tolerance, features as per the design document:\n\n- low latency per item read or written\n- high throughput, especially when writing an incoming stream of random items\n- ability to handle datasets much larger than RAM w/o degradation\n- crash friendliness, both in terms of fast recovery and not losing data\n- ease of backup and restore\n- a relatively simple, understandable (and thus supportable) code structure and data format\n- predictable behavior under heavy access load or large volume\n\nYou can find the original Bitcask Erlang implementation at: https://github.com/basho/bitcask\n\n## Roadmap\n\n- [x] Create new database\n- [ ] Open existing database\n- [x] Auto shard active file when it reaches the limit\n- [x] Verify lock for opening existing database\n- [x] List, Put, Get, Delete keys and values\n- [x] Default sane options and config\n- [x] Fold (map) function to entire key/value set\n- [ ] Flushable write buffer and Sync configuration\n- [ ] Search, Scan, Prefix, Indexing keys (Radix tree?)\n- [ ] Merge and compact database files\n- [ ] Create hint files for fast load times\n- [ ] Global time expiration of keys (per key expiration?)\n- [ ] Extensive unit testing and benchmarks\n- [ ] More...\n\n## API\n\n```go\n\nfunc Open(dirname string, config bitcask.Configuration) (*Bitcask, error)\n\nfunc (bc *Bitcask) List() []string\n\nfunc (bc *Bitcask) Put(key, value []byte) error\n\nfunc (bc *Bitcask) Get(key []byte) ([]byte, error)\n\nfunc (bc *Bitcask) Delete(key []byte) error\n\nfunc (bc *Bitcask) Fold(fn func([]byte, []byte) error) error\n\n```\n\n## Benchmarks\n\nSo far so good...\n\n```plain\ngoos: windows\ngoarch: amd64\npkg: github.com/phrozen/bitcask\ncpu: AMD Ryzen 7 5800X 8-Core Processor\nBenchmarkPut\nBenchmarkPut/128B\nBenchmarkPut/128B-16              579256              2043 ns/op          86.16 MB/s         208 B/op          2 allocs/op\nBenchmarkPut/256B\nBenchmarkPut/256B-16              545518              2124 ns/op         143.15 MB/s         352 B/op          2 allocs/op\nBenchmarkPut/1K\nBenchmarkPut/1K-16                425010              2689 ns/op         398.64 MB/s        1184 B/op          2 allocs/op\nBenchmarkPut/2K\nBenchmarkPut/2K-16                338184              3461 ns/op         605.64 MB/s        2336 B/op          2 allocs/op\nBenchmarkPut/4K\nBenchmarkPut/4K-16                239810              4916 ns/op         842.98 MB/s        4896 B/op          2 allocs/op\nBenchmarkPut/8K\nBenchmarkPut/8K-16                183066              6740 ns/op        1222.61 MB/s        9504 B/op          2 allocs/op\nBenchmarkPut/16K\nBenchmarkPut/16K-16               119998              9773 ns/op        1681.35 MB/s       18464 B/op          2 allocs/op\nBenchmarkPut/32K\nBenchmarkPut/32K-16                66092             16967 ns/op        1934.14 MB/s       40992 B/op          2 allocs/op\nPASS\nok      github.com/phrozen/bitcask      10.028s\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fphrozen%2Fbitcask","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fphrozen%2Fbitcask","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fphrozen%2Fbitcask/lists"}