{"id":15297415,"url":"https://github.com/ayaanhossain/sharedb","last_synced_at":"2025-04-13T23:16:06.268Z","repository":{"id":57466684,"uuid":"210664384","full_name":"ayaanhossain/ShareDB","owner":"ayaanhossain","description":"An on-disk pythonic embedded key-value store for compressed data storage and distributed data analysis","archived":false,"fork":false,"pushed_at":"2024-11-04T22:07:51.000Z","size":1846,"stargazers_count":7,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-13T23:15:57.001Z","etag":null,"topics":["data-analysis","data-storage","distributed","embedded-database","key-value","lmb","msgpack","multiprocessing","parallel","python","python-3","python-library","python-script","python2","python3","store"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ayaanhossain.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-09-24T17:50:14.000Z","updated_at":"2024-12-30T22:24:31.000Z","dependencies_parsed_at":"2022-09-19T08:00:19.755Z","dependency_job_id":null,"html_url":"https://github.com/ayaanhossain/ShareDB","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ayaanhossain%2FShareDB","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ayaanhossain%2FShareDB/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ayaanhossain%2FShareDB/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ayaanhossain%2FShareDB/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ayaanhossain","download_url":"https://codeload.github.com/ayaanhossain/ShareDB/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248794569,"owners_count":21162615,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-analysis","data-storage","distributed","embedded-database","key-value","lmb","msgpack","multiprocessing","parallel","python","python-3","python-library","python-script","python2","python3","store"],"created_at":"2024-09-30T19:17:22.229Z","updated_at":"2025-04-13T23:16:06.246Z","avatar_url":"https://github.com/ayaanhossain.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e\n    \u003ca href=\"https://github.com/ayaanhossain/ShareDB/\"\u003e\n        \u003cimg src=\"https://raw.githubusercontent.com/ayaanhossain/ShareDB/master/logo/logo.svg?sanitize=true\"  alt=\"ShareDB\" width=\"260\" class=\"center\"/\u003e\n    \u003c/a\u003e\n\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003e\n\t\u003ca href=\"https://github.com/ayaanhossain/ShareDB/actions\"\u003e\n\t    \u003cimg src=\"https://github.com/ayaanhossain/ShareDB/workflows/build/badge.svg\"\n\t     alt=\"CI-badge\"\u003e\n    \u003c/a\u003e\n\t\u003ca href=\"https://codecov.io/gh/ayaanhossain/ShareDB\"\u003e\n\t\t\u003cimg src=\"https://codecov.io/gh/ayaanhossain/ShareDB/branch/master/graph/badge.svg?token=syTKRG9H8O\"\n\t\t alt=\"codecov-badge\"\u003e\n    \u003c/a\u003e\n\t\u003ca href=\"https://pypi.org/project/ShareDB/\"\u003e\n\t\t\u003cimg src=\"https://img.shields.io/pypi/v/ShareDB\"\n\t\t alt=\"version-badge\"\u003e\n\t\u003c/a\u003e\n\t\u003ca href=\"https://pypi.org/project/ShareDB/\"\u003e\n\t    \u003cimg src=\"https://img.shields.io/pypi/pyversions/ShareDB\"\n\t     alt=\"python-badge\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://img.shields.io/badge/os-Linux-9cf\"\u003e\n\t    \u003cimg src=\"https://img.shields.io/badge/os-Linux-9cf\"\n\t     alt=\"os-badge\"\u003e\n    \u003c/a\u003e\n\t\u003ca href=\"./LICENSE\"\u003e\n\t    \u003cimg src=\"https://img.shields.io/pypi/l/ShareDB\"\n\t     alt=\"license-badge\"\u003e\n    \u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"#sharedb-in-action\"\u003eShareDB in Action\u003c/a\u003e •\n  \u003ca href=\"#installation\"\u003eInstallation\u003c/a\u003e •\n  \u003ca href=\"#license\"\u003eLicense\u003c/a\u003e •\n  \u003ca href=\"#contributing\"\u003eContributing\u003c/a\u003e •\n  \u003ca href=\"#acknowledgements\"\u003eAcknowledgements\u003c/a\u003e •\n  \u003ca href=\"https://github.com/ayaanhossain/ShareDB/blob/master/docs/API.md\"\u003eAPI\u003c/a\u003e\n\u003c/p\u003e\n\n`ShareDB` is a lightweight, **persistent key-value store** with a **dictionary-like interface** built on top of [LMDB](https://symas.com/lmdb/). It is intended to replace a python dictionary when\n\n 1. the key-value information needs to **persist locally** for later reuse,\n 2. the data needs to be **shared across multiple processes** with minimal overhead, and\n 3. the **keys** and **values** can be (de)serialized via **msgpack** or **pickle**.\n\nA `ShareDB` instance may be opened simultaneously in children, for reading in parallel, as long as a single process writes to the instance. **Parallel writes made across processes are not safe**; they are not guaranteed to be written, and may corrupt instance. `ShareDB` is primarily developed and tested using **Linux** and is compatible with both **Python 2.7 and Python 3.6 and above**.\n\n### `ShareDB` in Action\n```python\n\u003e\u003e\u003e from ShareDB import ShareDB           # Easy import\n\u003e\u003e\u003e print(ShareDB.__version__)            # Check version\n1.1.4\n\u003e\u003e\u003e myDB = ShareDB(path='./test.ShareDB') # Store ShareDB locally\n\u003e\u003e\u003e myDB['Name'] = ['Ayaan Hossain']      # Insert information\n\u003e\u003e\u003e myDB.get(key='Name')                  # Retrieve values\n['Ayaan Hossain']\n\u003e\u003e\u003e # Accelerated batch insertion/update via a single transaction\n\u003e\u003e\u003e len(myDB.multiset(kv_iter=zip(range(0, 10), range(10, 20))).sync())\n11\n\u003e\u003e\u003e 7 in myDB                             # Membership queries work\nTrue\n\u003e\u003e\u003e myDB['non-existent key']              # KeyError on invalid get as expected\nTraceback (most recent call last):\n...\nKeyError: \"key=non-existent key of \u003cclass 'str'\u003e is absent\"\n\u003e\u003e\u003e myDB.pop(7)                           # Pop a key just like a dictionary\n17\n\u003e\u003e\u003e list(myDB.multipopitem(num_items=5))  # Or, pop as many items as you need\n[(0, 10), (1, 11), (2, 12), (3, 13), (4, 14)]\n\u003e\u003e\u003e myDB.remove(5).remove(6).length()     # Chain removal of several keys\n2\n\u003e\u003e\u003e myDB.clear().length()                 # Or, clear entire ShareDB\n0\n\u003e\u003e\u003e myDB.drop()                           # Close/delete when you're done\nTrue\n```\n`ShareDB` methods either return data/result up on appropriate query, or a `self` is returned to facilitate method chaining. Terminal methods `.close()` and `.drop()` return a boolean indicating success.\n\nPlease see the `/examples/` directory for full examples of `ShareDB` usage.  Please see the [API.md](./docs/API.md) file for API details.\n\n### Installation\nOne-shot **installation/upgrade** of `ShareDB` from **PyPI**.\n```bash\n$ pip install --upgrade ShareDB\n```\nAlternatively, **clone** `ShareDB` from **GitHub**,\n```bash\n$ git clone https://github.com/ayaanhossain/ShareDB\n```\nnavigate into repo, and install via `pip`.\n```bash\n$ cd ShareDB\n$ pip install .\n```\nYou can **test** `ShareDB` with **pytest** inside the `/tests/` directory.\n```bash\n$ cd tests\n$ pytest\n```\n**Uninstallation** of `ShareDB` is easy with `pip`.\n```bash\n$ pip uninstall ShareDB\n```\n\n### License\n`ShareDB` (c) 2019-2024 Ayaan Hossain.\n\n`ShareDB` is an **open-source software** under [MIT](https://opensource.org/licenses/MIT) License.\n\nSee [LICENSE](./LICENSE) file for more details.\n\n### Contributing\nPlease **discuss** any issues/bugs you're facing, or any changes/features you have in mind by **opening an issue**, following the [Contributor Covenant](https://www.contributor-covenant.org/version/2/0/code_of_conduct). See [COC.md](./docs/COC.md) file for details. Please provide detailed **information**, and code **snippets** to facilitate debugging.\n\nTo contribute to `ShareDB`, please **clone** this repository, **commit** your code on a **separate new branch**, and **submit** a **pull request**. Please annotate and describe all **new** and **modified code** with detailed **comments** and **new unit tests** as applicable. Please ensure that modified builds **pass existing unit tests** before sending pull-requests.  For versioning, we use [SemVer](https://semver.org/).\n\n### Acknowledgements\n`ShareDB` is maintained by:\n\n - Ayaan Hossain | [github.com/ayaanhossain](https://github.com/ayaanhossain) | [@bioalgorithmist](https://twitter.com/bioalgorithmist)\n\n`ShareDB` was originally written to meet data analysis needs in [Prof. Howard Salis](https://twitter.com/hsalis)' Lab at [Penn State University](https://salislab.net/).\n\n### API\n`ShareDB` API details can be found in the [API.md](./docs/API.md) file.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fayaanhossain%2Fsharedb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fayaanhossain%2Fsharedb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fayaanhossain%2Fsharedb/lists"}