{"id":21959128,"url":"https://github.com/ldmtam/go-bitcask","last_synced_at":"2025-04-23T17:26:10.023Z","repository":{"id":214830681,"uuid":"737466257","full_name":"ldmtam/go-bitcask","owner":"ldmtam","description":"A Log-Structured Hash Table for Fast Key/Value Data implemented in Golang","archived":false,"fork":false,"pushed_at":"2024-01-21T04:30:31.000Z","size":58,"stargazers_count":14,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-30T01:51:43.028Z","etag":null,"topics":["bitcask","go","high-throughput","low-latency","storage-engine"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ldmtam.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-12-31T06:10:37.000Z","updated_at":"2025-01-07T16:56:45.000Z","dependencies_parsed_at":"2024-01-21T05:24:07.963Z","dependency_job_id":null,"html_url":"https://github.com/ldmtam/go-bitcask","commit_stats":null,"previous_names":["ldmtam/go-bitcask"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ldmtam%2Fgo-bitcask","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ldmtam%2Fgo-bitcask/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ldmtam%2Fgo-bitcask/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ldmtam%2Fgo-bitcask/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ldmtam","download_url":"https://codeload.github.com/ldmtam/go-bitcask/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250479293,"owners_count":21437338,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bitcask","go","high-throughput","low-latency","storage-engine"],"created_at":"2024-11-29T09:20:24.022Z","updated_at":"2025-04-23T17:26:10.005Z","avatar_url":"https://github.com/ldmtam.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# GO-BITCASK\n\n### What is Go-Bitcask\nA Log-Structured Hash Table for Fast Key/Value Data implemented in Golang\n\nGo-Bitcask storage engine features:\n1. Low latency per item read or written\n    - Write latency: ~80µs\n    - Read latency: ~17µs\n2. High throughput, especially when writing an incoming stream of random items\n    - Write throughput (per second): ~330_000\n    - Read throughput (per second): ~1_000_000\n3. Crash friendliness, both in terms of fast recovery and not losing data\n4. Ease of backup and restore\n5. Relatively simple, understandable code structure  and data format with high unit-test coverage (84%)\n\nFor more details, please reference to [Bitcask whitepaper](https://riak.com/assets/bitcask-intro.pdf)\n\n### Usage\nCreate new instance of **go-bitcask**\n```\ndb, err := gobitcask.New(\n    WithDirName(dirName),\n    WithSegmentSize(1024 * 1024 * 1024),  // 1 GB per segment file\n    WithMergeOpt(\u0026MergeOption{\n        Interval: 6 * time.Hour,          // run compaction every 6 hours\n        MinFiles: 5,                      // at least 5 data files before merging\n    })\n)\nif err != nil {\n    log.Fatalf(\"create gobitcask instance failed: %v\", err)\n}\ndefer db.Close()\n```\n\nStore key/value pair to storage\n```\nerr := db.Put([]byte(\"key1\"), []byte(\"val1\"))\nif err != nil {\n    log.Fatalf(\"store data to gobitcask failed: %v\", err)\n}\n```\n\nGet value from storage by particular key\n```\nval, err := db.Get([]byte(\"key1\"))\nif err != nil \u0026\u0026 err != gobitcask.ErrKeyNotFound {\n    log.Fatalf(\"get data from gobitcask failed: %v\", err)\n}\nfmt.Println(string(val))\n```\n\nDelete a key/value pair by key\n```\nerr := db.Delete([]byte(\"key1\"))\nif err != nil {\n    log.Fatalf(\"delete data from gobitcask failed: %v\", err)\n}\n```\n\nList all keys\n```\nkeys, err := db.ListKeys()\nif err != nil {\n    log.Fatalf(\"list keys from gobitcask failed: %v\", err)\n}\n```\n\nFold over all key/value pairs\n```\nerr := db.Fold(func(key, val []byte) error {\n    fmt.Printf(\"key: %v, val: %v\\n\", string(key), string(val))\n    return nil\n})\nif err != nil {\n    log.Fatalf(\"fold over all key/value pairs failed: %v\", err)\n}\n```\n\n### Benchmark\nMachine information: Macbook Pro 2021 (16 inch), M1 Pro, 16 GB RAM, 512 GB SSD\n\n| Action        | Num of keys | Key size (byte) | Value size (byte) | Duration (second) | RAM usage (MB)\n| ------------- | ----------- |---------------- |------------------ |-------------------|----------------\n| Put           | 1 000 000   |      8          |        128        |       2.809       |      636\n| Put           | 1 000 000   |      16         |        512        |       3.331       |      1514\n| Put           | 10 000 000  |      8          |        128        |       30.047      |      5697\n| Random Get    | 1 000 000   |      8          |        128        |       0.769       |      N/A\n| Random Get    | 1 000 000   |      16         |        512        |       0.867       |      N/A\n| Random Get    | 10 000 000  |      8          |        128        |       8.81        |      N/A","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fldmtam%2Fgo-bitcask","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fldmtam%2Fgo-bitcask","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fldmtam%2Fgo-bitcask/lists"}