{"id":17946589,"url":"https://github.com/vyuham/dstore","last_synced_at":"2025-03-24T20:32:49.460Z","repository":{"id":49211043,"uuid":"328693451","full_name":"vyuham/dstore","owner":"vyuham","description":"A partially distributed storage framework, using the two-layer architecture, in-development.","archived":false,"fork":false,"pushed_at":"2021-06-23T12:25:48.000Z","size":920,"stargazers_count":4,"open_issues_count":0,"forks_count":3,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-10-29T07:06:34.887Z","etag":null,"topics":["grpc","key-value-database","rust-lang"],"latest_commit_sha":null,"homepage":"https://vyuham.github.io/dstore","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vyuham.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-01-11T14:39:32.000Z","updated_at":"2022-08-21T10:59:12.000Z","dependencies_parsed_at":"2022-09-26T20:10:34.903Z","dependency_job_id":null,"html_url":"https://github.com/vyuham/dstore","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vyuham%2Fdstore","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vyuham%2Fdstore/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vyuham%2Fdstore/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vyuham%2Fdstore/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vyuham","download_url":"https://codeload.github.com/vyuham/dstore/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245348328,"owners_count":20600627,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["grpc","key-value-database","rust-lang"],"created_at":"2024-10-29T07:06:38.051Z","updated_at":"2025-03-24T20:32:49.054Z","avatar_url":"https://github.com/vyuham.png","language":"Rust","readme":"\u003cp align=\"center\"\u003e\u003cimg src=\"docs/dstore.png\" width=\"350px\"/\u003e\u003c/p\u003e\n\ndstore is a partially distributed storage framework inspired from the [FAASM project](https://github.com/faasm).\n\n## Database\nBuilt as an in-memory datastore with dual layer architecture that enables the storage of Key-Value pairs on nodes in coherence with a single **'Global'** datastore for synchronization purposes. Keys are UIDs and should technically only be associated with **a** Value for eternity. Hopefully that will be implemented with the hash of said Value being used for the same.\n\n## Architecture\n\u003cp align=\"center\"\u003e\u003cimg src=\"docs/system.png\" width=\"350px\"/\u003e\u003c/p\u003e\n\nThe two layer architecture consists of an in-memory datastore on each node called **'Local'**, which stores copies of data-objects as Key-Value pairs where the Key acts as a UID and enables the user to have a single copy of the data on each node, thus decreasing network access to access the same.\n\n## Demo\nTo demo the working of Dstore, we have created a CLI based Key-Value database that isn't fully featured, to try it out, follow the steps given below:\n1. Download the code:\n```bash\ngit clone https://github.com/vyuham/dstore\n```\n2. Run the server(ensure `cargo` is installed with [rustup](https://rustup.rs)):\n```bash\ncargo run --example global\n```\nEnsure the following is showing on the terminal:\n```\nDstore server listening on 127.0.0.1:50051\n```\n3. On another terminal, start a client:\n```bash\ncargo run --example local\n```\nEnsure the following is showing on the terminal:\n```\ndstore v0.1.0 (addr: 127.0.0.1:50052)\nThis is an experimental database, do contribute to further developments at https://github.com/vyuham/dstore. \nUse `.exit` to exit the repl\ndb \u003e \n```\nNow you know some of the workings of dstore :D\n\n### REPL Commands\nThe demo's REPL uses commands similar to [KVDB](https://github.com/de-sh/vyuham), you may use the following to interact with it:\n- **SET:** The user/process can store data on the system using the `SET`/`PUT` command word, as shown below:\n```\ndb\u003e SET key value\n```\nThe output `Database updated` implies that `key` now maps to `value` within dstore. When you try to `SET` a key that already exists in 'Local', expect to get the output:\n```\nKey occupied!\n```\nWhen a key exists only on 'Global' it is updated on local and the `SET` operation is abandoned, with the message:\n```\n(Updated local) Key occupied!\n```\n- **GET:** The user/process can access data from the system using the `GET`/`OUT` command word, as shown below:\n```\ndb\u003e GET key\n```\nThis will output:\n```\ndb: key -\u003e value\n```\nThe above output implies that `key` is mapped to `value` within dstore. If you try to `GET` a random key which isn't mapped to any value in the system, you will recieve the output:\n```\nKey-Value mapping doesn't exist\n```\nWhen you try to `GET` a key(say `hi`) that isn't available in 'Local' but exists in 'Global'(associated with a value `hello`), expect to receve the output:\n```\nglobal: hi -\u003e hello     (Updating Local)\n```\n- **DEL:** The user/process can removes local as well as global\n copies of a Key-Value pair with the `DEL`/`REM` keyword as shown below:\n```\ndb\u003e DEL key\n```\nThis might only remove the key-value mapping from within the current 'Local' and 'Global' while other 'Local's may still maintain a copy as currently the `REPL` thread blocks the `update()` thread leading to a possible lack of support for invalidating KEYs that aren't unique, at all times. Discussion on how to reflect changes across all 'Local's needs to take place. To test these claims, please change the port number of `local_addr` in 'example/local.rs` and try opening a few 'Local' clients, with a unique port assigned to each and do different order of tasks relating to deleting key-value pairs.\n\n## Contribution\nPlease consider opening an issue to discuss possible feature additions and use cases for the framework. We would love to review genuine PRs to fix bugs, solve issues and add feature, so do consider opening some!\n\nAs always, maintain decorum and follow the [rust-lang code of conduct](https://www.rust-lang.org/policies/code-of-conduct). Thank you!","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvyuham%2Fdstore","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvyuham%2Fdstore","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvyuham%2Fdstore/lists"}