{"id":13672387,"url":"https://github.com/raptazure/unifier","last_synced_at":"2025-04-27T21:32:58.543Z","repository":{"id":56026994,"uuid":"278783836","full_name":"raptazure/unifier","owner":"raptazure","description":"A multi-threaded, persistent key/value store","archived":false,"fork":false,"pushed_at":"2021-02-09T14:24:04.000Z","size":303,"stargazers_count":17,"open_issues_count":2,"forks_count":2,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-08-04T05:01:29.437Z","etag":null,"topics":["database","key-value-store"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/raptazure.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-07-11T03:57:28.000Z","updated_at":"2024-08-04T05:01:29.438Z","dependencies_parsed_at":"2022-08-15T11:40:44.290Z","dependency_job_id":null,"html_url":"https://github.com/raptazure/unifier","commit_stats":null,"previous_names":["raptazure/unity.kv"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/raptazure%2Funifier","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/raptazure%2Funifier/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/raptazure%2Funifier/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/raptazure%2Funifier/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/raptazure","download_url":"https://codeload.github.com/raptazure/unifier/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251212827,"owners_count":21553528,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","key-value-store"],"created_at":"2024-08-02T09:01:34.122Z","updated_at":"2025-04-27T21:32:58.250Z","avatar_url":"https://github.com/raptazure.png","language":"Rust","funding_links":[],"categories":["Rust"],"sub_categories":[],"readme":"\n\u003cp align=\"center\"\u003e\n    \u003cimg width=\"180\" alt=\"Logo\" src=\"extra/logo.png\"\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n    A multi-threaded, persistent key/value store server and client with networking over a custom protocol.\n\u003c/p\u003e\n\n## Quick start\n\n- Build: `cargo build`\n- Run server: `unifier-server -h`\n  ```\n  unifier-server 0.1.0\n\n  USAGE:\n    unifier-server [OPTIONS]\n\n  FLAGS:\n      -h, --help       Prints help information\n      -V, --version    Prints version information\n\n  OPTIONS:\n          --addr \u003cIP:PORT\u003e          Sets the listening address [default: 127.0.0.1:4000]\n          --engine \u003cENGINE-NAME\u003e    Sets the storage engine [possible values: kvs, sled]\n  ```\n  Note: If `--engine` is specified, then `ENGINE-NAME` must be either \"kvs\", in which\n  case the built-in engine is used, or \"sled\", in which case sled is used. If\n  this is the first run (there is no data previously persisted) then the default\n  value is \"kvs\"; if there is previously persisted data then the default is the\n  engine already in use. If data was previously persisted with a different\n  engine than selected, it will print an error.\n- Run client: `unifier-client -h`\n  ```\n  unifier-client 0.1.0\n\n\tUSAGE:\n    unifier-client \u003cSUBCOMMAND\u003e\n\n  FLAGS:\n      -h, --help       Prints help information\n      -V, --version    Prints version information\n\n  SUBCOMMANDS:\n      get    Get the string value of a given string key\n      rm     Remove a given string key\n      set    Set the value of a string key to a string\n  ```\n\n## Why unifier (unity.kv)?\n\n- Multi-threaded: many threads are created within a process, executing independently but concurrently sharing process resources to finish tasks in a much faster way. This efficiency comes from the unity of threads.\n- Traits for intergration:\n  - Two key-value store engines. `KvsEngine` trait defines the storage interface. `KvStore` implements `KvsEngine` for the `kvs` storage engine and `SledKvsEngine` implements `KvsEngine` for the `sled` storage engine.  \n  - Three threadpool implementations. `ThreadPool` trait contains methods that create a new thread pool to spawn the specified number of threads, and that spawn a function into the threadpool. `NaiveThreadPool` implements `ThreadPool` and spawns a new thread every time the `spawn` method is called. `RayonThreadPool` implements `ThreadPool` using a data parallelism library called [rayon](https://github.com/rayon-rs/rayon). And `SharedQueueThreadPool` implements `ThreadPool` using a shared queue.\n  - Different kinds of engines and threadpools to choose is the unity of implementations.\n- Built on top of open-source projects and online tutorials, this is the unity of crates and experiences.\n- I have been playing _Assassin's Creed Unity_ recently.  :)\n\n## Benchmark\n\nCompared with [sled](https://github.com/spacejam/sled) (a concurrent embedded key-value database), `kvs` engine takes samller space and offers faster speed. The benchmark results are as follows:\n\n\u003cbr/\u003e\n\n\u003cimg src=\"extra/line.png\" alt=\"line\"\u003e\n\u003cimg src=\"extra/kvs_8.png\" alt=\"kvs_8\"\u003e\n\u003cimg src=\"extra/sled_8.png\" alt=\"sled_8\"\u003e\n\u003cimg src=\"extra/kvs_16.png\" alt=\"kvs_16\"\u003e\n\u003cimg src=\"extra/sled_16.png\" alt=\"sled_16\"\u003e\n\n\n## Testing\n`cargo test --test cli`\n- [x] server_cli_version\n- [x] client_cli_version\n- [x] client_cli_no_args\n- [x] client_cli_invalid_subcommand\n- [x] test client_cli_invalid_get\n- [x] client_cli_invalid_rm\n- [x] client_cli_invalid_set\n- [x] cli_log_configuration\n- [x] cli_wrong_engine\n- [x] cli_access_server_kvs_engine\n- [x] cli_access_server_sled_engine\n\n`cargo test --test kv_store`\n- [x] remove_non_existent_key\n- [x] remove_key\n- [x] get_non_existent_value\n- [x] get_stored_value\n- [x] overwrite_value\n- [x] concurrent_get\n- [x] concurrent_set\n- [x] compaction\n\n`cargo test --test thread_pool`\n- [x] naive_thread_pool_spawn_counter\n- [x] shared_queue_thread_pool_spawn_counter\n- [x] rayon_thread_pool_spawn_counter\n- [x] shared_queue_thread_pool_panic_task\n\n\u003cbr /\u003e\n\n\u003e Thanks to https://github.com/pingcap/talent-plan\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fraptazure%2Funifier","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fraptazure%2Funifier","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fraptazure%2Funifier/lists"}