{"id":19499433,"url":"https://github.com/prxssh/beetle","last_synced_at":"2026-05-07T18:09:14.264Z","repository":{"id":272165227,"uuid":"909397029","full_name":"prxssh/beetle","owner":"prxssh","description":"redis compliant key-value store using bitcask 🐞","archived":false,"fork":false,"pushed_at":"2025-04-27T05:26:40.000Z","size":178,"stargazers_count":1,"open_issues_count":10,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-27T06:26:20.186Z","etag":null,"topics":["bitcask","cache","database","elixir","redis"],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/prxssh.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-12-28T15:26:22.000Z","updated_at":"2025-03-15T17:44:26.000Z","dependencies_parsed_at":"2025-01-12T16:37:17.237Z","dependency_job_id":"2fd82384-c9c3-4872-820c-eef771418066","html_url":"https://github.com/prxssh/beetle","commit_stats":null,"previous_names":["prxssh/beetle"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prxssh%2Fbeetle","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prxssh%2Fbeetle/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prxssh%2Fbeetle/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prxssh%2Fbeetle/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/prxssh","download_url":"https://codeload.github.com/prxssh/beetle/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prxssh%2Fbeetle/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":257471961,"owners_count":22550733,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bitcask","cache","database","elixir","redis"],"created_at":"2024-11-10T22:04:08.996Z","updated_at":"2026-05-07T18:09:09.237Z","avatar_url":"https://github.com/prxssh.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Beetle\n\nBeetle is a Elixir implementation of [Bitcask by Riak](https://riak.com/assets/bitcask-intro.pdf) \npaper and aims to closely follow the spec.\n\nBitcask is one of the most efficient embedded key-value database designed to\nhandle production-grade traffic. It uses a log-structured hash table for fast\nkey-value data which, in a simple language, means that the data will be written\nsequentially to an append-only log file and there will be pointers for each\n`key` pointing to the `position` of its log entry.\n\n## Benefits of this approach\n\n- Low latency for read and write operations\n- High Write Throughput\n- Single disk seek to retrieve any value\n- Predictable lookup and insert performance\n- Crash recovery is fast and bounded\n- Backing up is easy - Just copying the directory would suffice\n\n## Limitations\n\nThe main limitation is that all the keys must fit in RAM since they're held\ninside das an in-memory hash table. This adds a huge constraint on the system\nthat it needs to have enough memory to contain the entire keyspace along with\nother essentials like Filesystem buffers. Although this weakness seems a major\none but the solution to this is fairly simple. We can typically shard the keys\nand scale it  horizontally without losing much of the basic operations like\nCreate, Read, Update, and Delete.\n\n## Roadmap and Status\n\nThe high-level ambitious plan for the project, in order:\n\n|  #  | Step                                                      | Status |\n| :-: | --------------------------------------------------------- | :----: |\n|  1  | Bitcask Implementation                                    |   ✅   |\n|  2  | Redis Serialization Protocol                              |   ✅   |\n|  3  | Basic commands like GET, SET, DEL                         |   ✅   |\n|  4  | Redis Transactions \u0026 Command Pipelining                   |   ✅   |\n|  5  | Support database sharding                                 |   ✅   |\n|  6  | Support strings datatype                                  |   ✅   |\n|  7  | Support list datatype                                     |   ❌   |\n|  8  | Support hash datatype                                     |   ❌   |\n|  9  | Support bloom filter datatype                             |   ❌   |\n|  10 | Support bitmap datatype                                   |   ❌   |\n|  11 | Support pubsub                                            |   ❌   |\n|  12 | Test cases for the modules                                |   ❌   |\n|  13 | Make it distributed using Raft consensus algorithm        |   ❌   |\n\nThe available commands can be found [here](lib/beetle/command/mapping.ex).\n\n## Get started\n\n### Setting up Beetle with Docker\n\nThe easiest way to get started with Beetle is using [Docker](https://www.docker.com/) by running the following command:\n\n\n```bash\ndocker build -t beetle .\n\ndocker run -p 6969:6969 --name beetle beetle\n```\n\nThe above commands will build the docker image and start Beetle server running\nlocally on the port `6969` and you can connect to it using \n[Redis CLI](https://redis.io/docs/latest/develop/tools/cli/)\n\n### Running the application locally\n\nTo run Beetle locally either for development or just playing around, you need\nto have the [Elixir](https://elixir-lang.org/) installed.\n\n```bash\ngit clone https://github.com/prxssh/beetle\ncd beetle\niex -S mix run\n```\n\nThis will start the Beetle server on port `6969`. \n\nAdditonally you can also run Beetle by providing a configuration file:\n\n```bash\niex -S mix run -- config example/beetle.conf\n```\n\n## Benchmarks \n\nPerformance benchmarks on M4 Macbook Pro using [Redis benchmark](https://redis.io/docs/latest/operate/oss_and_stack/management/optimization/benchmarks/)\ntool with 100K operations across 10 million unique keys:\n\n### Get Operations\n\n```bash\n$ redis-benchmark -h localhost -p 6969 -n 100000 -r 10000000 -c 100 -t get\n\nSummary:\n  throughput summary: 136798.91 requests per second\n  latency summary (msec):\n          avg       min       p50       p95       p99       max\n        0.433     0.136     0.407     0.519     1.271     1.751\n```\n\n### Set Operations\n\nTested with 100 concurrent clients\n\n```bash\n$ redis-benchmark -h localhost -p 6969 -n 100000 -r 10000000 -c 100 -t set\n\nSummary:\n  throughput summary: 137362.64 requests per second\n  latency summary (msec):\n          avg       min       p50       p95       p99       max\n        0.428     0.128     0.407     0.487     1.255     1.559\n```\n\n### Performance Summary\n\n- Sustained throughput of 136K+ operations per second\n- Sub-millisecond average latency for both read and write operations\n- P99 latency under 1.3ms for reads and writes\n- Consistent performance across large keyspace (10M unique keys)\n\n## References\n\n- [Bitcask paper](https://riak.com/assets/bitcask-intro.pdf)\n- [Redis Serialization Protocol Specification](https://redis.io/docs/latest/develop/reference/protocol-spec/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprxssh%2Fbeetle","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprxssh%2Fbeetle","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprxssh%2Fbeetle/lists"}