{"id":18821612,"url":"https://github.com/octu0/bitcaskdb","last_synced_at":"2025-04-14T00:32:41.709Z","repository":{"id":36958140,"uuid":"502509338","full_name":"octu0/bitcaskdb","owner":"octu0","description":"high performance KV database based on bitcask","archived":false,"fork":false,"pushed_at":"2023-05-12T05:28:42.000Z","size":604,"stargazers_count":9,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"master","last_synced_at":"2024-06-26T11:43:12.998Z","etag":null,"topics":["bitcask","database","golang","kvdb","lsm","natsio"],"latest_commit_sha":null,"homepage":"https://pkg.go.dev/github.com/octu0/bitcaskdb","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/octu0.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-06-12T03:27:36.000Z","updated_at":"2023-10-21T10:50:14.000Z","dependencies_parsed_at":"2024-06-19T22:51:49.448Z","dependency_job_id":"5afcf4f2-2e9f-4d51-bf9d-69f3f7580018","html_url":"https://github.com/octu0/bitcaskdb","commit_stats":null,"previous_names":[],"tags_count":20,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/octu0%2Fbitcaskdb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/octu0%2Fbitcaskdb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/octu0%2Fbitcaskdb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/octu0%2Fbitcaskdb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/octu0","download_url":"https://codeload.github.com/octu0/bitcaskdb/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223612891,"owners_count":17173631,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bitcask","database","golang","kvdb","lsm","natsio"],"created_at":"2024-11-08T00:44:53.846Z","updated_at":"2024-11-08T00:44:54.711Z","avatar_url":"https://github.com/octu0.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# bitcaskdb\n\n[![MIT License](https://img.shields.io/github/license/octu0/bitcaskdb)](https://github.com/octu0/bitcaskdb/blob/master/LICENSE)\n[![GoDoc](https://godoc.org/github.com/octu0/bitcaskdb?status.svg)](https://godoc.org/github.com/octu0/bitcaskdb)\n[![Go Report Card](https://goreportcard.com/badge/github.com/octu0/bitcaskdb)](https://goreportcard.com/report/github.com/octu0/bitcaskdb)\n[![Releases](https://img.shields.io/github/v/release/octu0/bitcaskdb)](https://github.com/octu0/bitcaskdb/releases)\n\nOriginal code is [bitcask](https://git.mills.io/prologic/bitcask) and `bitcaskdb` modifies I/O operations and implement replication.  \nSmall Value are still operated in memory, Large Value are directly I/O operation on disk.  \nThis makes it possible to perform Merge operations and large data store with minimal RAM utilization.\n\nA high performance Key/Value store written in [Go](https://golang.org) with a predictable read/write performance and high throughput. \nUses a [Bitcask](https://en.wikipedia.org/wiki/Bitcask) on-disk layout (LSM+WAL) similar to [Riak](https://riak.com/).\n\n## Installation\n\n```shell\ngo get github.com/octu0/bitcaskdb\n```\n\n## Example\n\n`bitcaskdb` methods are implemented to use [io.Reader](https://pkg.go.dev/io#Reader) / [io.ReadCloser](https://pkg.go.dev/io#ReadCloser), etc.\n\n```go\nimport (\n\t\"bytes\"\n\t\"io\"\n\t\"fmt\"\n\n\t\"github.com/octu0/bitcaskdb\"\n)\n\nfunc main() {\n\tdb, err := bitcaskdb.Open(\"./data/mydb\")\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tdefer db.Close()\n\n\t// PutBytes() can be set using byte slice\n\tdb.PutBytes([]byte(\"hello\"), []byte(\"world\"))\n\n\t// Get() returns io.ReadCloser\n\tr, err := db.Get([]byte(\"hello\"))\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tdefer r.Close()\n\n\tdata, _ := io.ReadAll(r)\n\n\t// Put() can be specify io.Reader\n\tdb.Put([]byte(\"foo\"), bytes.NewReader([]byte(\"very large data...\")))\n\n\t// PutWithTTL()/PutBytesWithTTL() can be set to data with expiration time\n\tdb.PutWithTTL([]byte(\"bar\"), bytes.NewReader(data), 10*time.Second)\n\n\t// Sync() flushes all buffers to disk\n\tdb.Sync()\n\n\tr, err := db.Get([]byte(\"foo\"))\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tdefer r.Close()\n\n\thead := make([]byte, 4)\n\tr.Read(head)\n\n\t// Delete() can delete data with key\n\tdb.Delete([]byte(\"foo\"))\n\n\t// RunGC() deletes all expired keys\n\tdb.RunGC()\n\n\t// Merge() rebuilds databases and reclaims disk space\n\tdb.Merge()\n}\n```\n\n## Benchmark\n\n`bitcaskdb` is tuned for larger sizes of Value, in particular there is a major improvement for inputs and outputs using [io.Reader](https://pkg.go.dev/io#Reader).  \nDefault Buffer size is aligned to 128KB, this value can be changed with [runtime.Context](https://pkg.go.dev/github.com/octu0/bitcaskdb#WithRuntimeContext).\n\n```\ngoos: darwin\ngoarch: amd64\npkg: github.com/octu0/bitcaskdb\ncpu: Intel(R) Core(TM) i7-8569U CPU @ 2.80GHz\n\nBenchmarkGet\nBenchmarkGet/prologic/bitcask/128B\nBenchmarkGet/prologic/bitcask/128B-8      1317308       897.5 ns/op   142.62 MB/s       160 B/op     1 allocs/op\nBenchmarkGet/prologic/bitcask/256B\nBenchmarkGet/prologic/bitcask/256B-8      1229084       973.1 ns/op   263.08 MB/s       288 B/op     1 allocs/op\nBenchmarkGet/prologic/bitcask/128K\nBenchmarkGet/prologic/bitcask/128K-8        34060       30690 ns/op  4270.86 MB/s    139264 B/op     1 allocs/op\nBenchmarkGet/prologic/bitcask/256K\nBenchmarkGet/prologic/bitcask/256K-8        18928       56895 ns/op  4607.53 MB/s    270337 B/op     1 allocs/op\nBenchmarkGet/prologic/bitcask/512K\nBenchmarkGet/prologic/bitcask/512K-8         9092      130977 ns/op  4002.91 MB/s    532483 B/op     1 allocs/op\nBenchmarkGet/octu0/bitcaskdb/128B\nBenchmarkGet/octu0/bitcaskdb/128B-8        102139       54958 ns/op     2.33 MB/s    205145 B/op    21 allocs/op\nBenchmarkGet/octu0/bitcaskdb/256B\nBenchmarkGet/octu0/bitcaskdb/256B-8         35295       45820 ns/op     5.59 MB/s    206286 B/op    21 allocs/op\nBenchmarkGet/octu0/bitcaskdb/128K\nBenchmarkGet/octu0/bitcaskdb/128K-8         32757       40773 ns/op  3214.66 MB/s    199695 B/op    21 allocs/op\nBenchmarkGet/octu0/bitcaskdb/256K\nBenchmarkGet/octu0/bitcaskdb/256K-8         25388       49611 ns/op  5284.03 MB/s    214647 B/op    21 allocs/op\nBenchmarkGet/octu0/bitcaskdb/512K\nBenchmarkGet/octu0/bitcaskdb/512K-8         18439       69490 ns/op  7544.78 MB/s    215778 B/op    21 allocs/op\n\nBenchmarkPut\nBenchmarkPut/prologic/bitcask/WithNosync/128B\nBenchmarkPut/prologic/bitcask/WithNosync/128B-8       85623       13452 ns/op     9.52 MB/s     41 B/op     2 allocs/op\nBenchmarkPut/prologic/bitcask/WithNosync/256B\nBenchmarkPut/prologic/bitcask/WithNosync/256B-8       69417       21407 ns/op    11.96 MB/s     43 B/op     2 allocs/op\nBenchmarkPut/prologic/bitcask/WithNosync/128K\nBenchmarkPut/prologic/bitcask/WithNosync/128K-8         140     8566745 ns/op    15.30 MB/s   1569 B/op     9 allocs/op\nBenchmarkPut/prologic/bitcask/WithNosync/256K\nBenchmarkPut/prologic/bitcask/WithNosync/256K-8         100    16770433 ns/op    15.63 MB/s   3052 B/op    16 allocs/op\nBenchmarkPut/prologic/bitcask/WithNosync/512K\nBenchmarkPut/prologic/bitcask/WithNosync/512K-8         100    34380284 ns/op    15.25 MB/s   6193 B/op    31 allocs/op\nBenchmarkPut/octu0/bitcaskdb/WithNosync/128B\nBenchmarkPut/octu0/bitcaskdb/WithNosync/128B-8       333992        3548 ns/op    36.07 MB/s    200 B/op     9 allocs/op\nBenchmarkPut/octu0/bitcaskdb/WithNosync/256B\nBenchmarkPut/octu0/bitcaskdb/WithNosync/256B-8       305065        3900 ns/op    65.64 MB/s    208 B/op    10 allocs/op\nBenchmarkPut/octu0/bitcaskdb/WithNosync/128K\nBenchmarkPut/octu0/bitcaskdb/WithNosync/128K-8        10000      242319 ns/op   540.91 MB/s    594 B/op    10 allocs/op\nBenchmarkPut/octu0/bitcaskdb/WithNosync/256K\nBenchmarkPut/octu0/bitcaskdb/WithNosync/256K-8         7059      447583 ns/op   585.69 MB/s    461 B/op    10 allocs/op\nBenchmarkPut/octu0/bitcaskdb/WithNosync/512K\nBenchmarkPut/octu0/bitcaskdb/WithNosync/512K-8         3835      812899 ns/op   644.96 MB/s    848 B/op    10 allocs/op\n```\n\n## License\n\nMIT, see LICENSE file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foctu0%2Fbitcaskdb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foctu0%2Fbitcaskdb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foctu0%2Fbitcaskdb/lists"}