{"id":26787101,"url":"https://github.com/thenoobgrammer/bitcask-db-impl","last_synced_at":"2025-10-14T19:07:40.809Z","repository":{"id":280087257,"uuid":"940613901","full_name":"thenoobgrammer/bitcask-db-impl","owner":"thenoobgrammer","description":"A fun bitcask implementation in go, to gain more understanding on mutexes and log-structured hash tables","archived":false,"fork":false,"pushed_at":"2025-03-04T07:09:41.000Z","size":9,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-10-14T19:04:39.801Z","etag":null,"topics":["bitcask","golang"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thenoobgrammer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-02-28T13:42:33.000Z","updated_at":"2025-03-16T05:18:08.000Z","dependencies_parsed_at":"2025-08-04T21:34:41.363Z","dependency_job_id":null,"html_url":"https://github.com/thenoobgrammer/bitcask-db-impl","commit_stats":null,"previous_names":["thenoobgrammer/bitcask-db-impl"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/thenoobgrammer/bitcask-db-impl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thenoobgrammer%2Fbitcask-db-impl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thenoobgrammer%2Fbitcask-db-impl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thenoobgrammer%2Fbitcask-db-impl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thenoobgrammer%2Fbitcask-db-impl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thenoobgrammer","download_url":"https://codeload.github.com/thenoobgrammer/bitcask-db-impl/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thenoobgrammer%2Fbitcask-db-impl/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279020642,"owners_count":26086895,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-14T02:00:06.444Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bitcask","golang"],"created_at":"2025-03-29T12:18:23.577Z","updated_at":"2025-10-14T19:07:40.793Z","avatar_url":"https://github.com/thenoobgrammer.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"### Bitcask implementation\n\nThis is my attempt to understand how bitcask implementation work.\n\nYou can find more information about it [here](https://riak.com/assets/bitcask-intro.pdf)\n\n### Initial take\n\nWhat I intend to do first, is to be able to write some entries to a file, with a capped size at `1MB` for each data file.\nThe purpose is to emulate an HDD-To perform write operations on a hard drive we are following the _append-only_ process, where we only add entries and run a compactor at random during runtime to clear the deleted and corrupted data.\n\n### Opening a directory\n\nWhen we `open` a directory, we first read **all files** in order to get some information to build our hash table (key/dir map).\n\nWe do this by looping through each file, extract the entries at the same time we retrieve the files information-This will allow us to understand the size of the file and the offset (last byte position).\n\nThe **active file** is set on the first file we find with less occupied space than the capped amount, e.g\n\n```bash\nbitcask-001.data # 1 MB \u003c-- full\nbitcask-002.data # 1 MB \u003c-- full\nbitcask-003.data # 320 KB \u003c-- active file\n```\n\n###### Offset\n\nThe offset is the **byte position** for the last added entry. This encourages fast processing for our next reads.\n\n```\n0 key=\"name\",value=\"elie\"\n29 key=\"name\",value=\"karl\"\n88 key=\"name\",value=\"ralph\" // \u003c-- The offset for this file is 88\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthenoobgrammer%2Fbitcask-db-impl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthenoobgrammer%2Fbitcask-db-impl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthenoobgrammer%2Fbitcask-db-impl/lists"}