{"id":18772863,"url":"https://github.com/chux0519/toy-kv","last_synced_at":"2025-12-13T09:30:17.758Z","repository":{"id":108097638,"uuid":"168526628","full_name":"chux0519/toy-kv","owner":"chux0519","description":"A simple kv server","archived":false,"fork":false,"pushed_at":"2019-02-11T09:06:32.000Z","size":387,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2024-12-29T08:23:49.584Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chux0519.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-01-31T13:07:30.000Z","updated_at":"2019-02-11T09:06:34.000Z","dependencies_parsed_at":null,"dependency_job_id":"25a2f319-6fed-47da-85d3-f01a0fbc1f4e","html_url":"https://github.com/chux0519/toy-kv","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chux0519%2Ftoy-kv","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chux0519%2Ftoy-kv/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chux0519%2Ftoy-kv/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chux0519%2Ftoy-kv/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chux0519","download_url":"https://codeload.github.com/chux0519/toy-kv/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239681024,"owners_count":19679508,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T19:31:18.652Z","updated_at":"2025-12-13T09:30:17.723Z","avatar_url":"https://github.com/chux0519.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Toy-KV\n\nA single node kv server using key-value separations strategy instead of LSM-based technology.\n\nPS: Do not use this in any production environment.\n\n## Features\n\n- SSD Friendly(No huge write amplification)\n- Data Persistence\n- Easy to Integrate (you can change the transport layer easily)\n\n## Usage\n\nToy-kv provides a store class that supports `Get`, `Put`, `Delete` and `Scan` operations.\n\nIt is suggested start with a simple C/S demo.\n\n### Server\n\nDirect IO may only support in Linux based OS, so using docker to start a server. (It also works on OSX Mojave, but no guarantee)\n\n\u003e mkdir -p ./toydb\n\u003e\n\u003e docker run -it -e SERVER_PORT=8888 -e DB_DIR=/toydb -p 8888:8888 -v \"$(pwd)\"/toydb:/toydb chux0519/toy_server\n\nAbove will start a local kv server at port 8888, and mount `./toydb` to container.\n\nAlso, you can run following on your local machine.\n\n\u003e mkdir -p ./toydb\n\u003e\n\u003e SERVER_PORT=8888 DB_DIR=\"$(pwd)\"/toydb cargo run --example server\n\n### Client\n\n\u003e git clone https://github.com/chux0519/toy-kv \u0026\u0026 cd toy-kv\n\u003e\n\u003e cargo run --example client 127.0.0.1:8888\n\nNow, play with the client.\n\n![demo](imgs/demo.png)\n\n## Design\n\nStore kv pair with three separate files: keys, values, and buffer.\nSo `Store` has three components: `Keys`, `Values` and `Buffer` (When running a store instance, it also build the `Index` of `Values` from `Keys`).\n\n### Keys\n\nThe keys file ends with `.k`\n\nEach keys file may contain several sections, and each section includes 65536 records, each record is in the format: `[8bytes][4bytes]`. The front part of a record is for the key, and the second half stores the position of relevant value.\n\nWhen creating a `Store`, it read the `.k` file, and build the index for all values. Then the last section of keys will be mapped to memory(mmap).\n\n### Values\n\nThe values file ends with `.v`\n\nValues files' format is similar to the `.k` files, it contains several sections, and each section contains 65536 records, unlike keys file, each record of values file is entirely for value(256 bytes).\n\nA Store instance only holds the file handle of the `.v` file(opening with O_DIRECT flag), so it uses direct io rather than other buffer io methods.\n\n### Buffer\n\nEnds with `.b`, its size is fixed at 16mb (the same as each section of values).\nWhen the store does `Put` action, it first write data to the buffer, when the buffer is full, then it will flush to values ​​file using direct io.\n\n## Limitation\n\n- Single thread usage\n- Only support  fixed length kv pair, (8 bytes, 256 bytes)\n- Only support at most 0xffffffff(1\u003c\u003c32) kv pairs (using 4 bytes to index value)\n\n## TODOS\n\n- [ ] More reasonable benchmark (YCSB support maybe)\n- [ ] Multi-thread usage and multi-shard storage\n- [ ] Garbage collection","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchux0519%2Ftoy-kv","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchux0519%2Ftoy-kv","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchux0519%2Ftoy-kv/lists"}