{"id":18373049,"url":"https://github.com/coleifer/python-lsm-db","last_synced_at":"2025-03-29T16:02:26.166Z","repository":{"id":35826666,"uuid":"40109535","full_name":"coleifer/python-lsm-db","owner":"coleifer","description":"Python bindings for the SQLite4 LSM database.","archived":false,"fork":false,"pushed_at":"2025-03-11T14:39:39.000Z","size":1404,"stargazers_count":130,"open_issues_count":0,"forks_count":16,"subscribers_count":11,"default_branch":"master","last_synced_at":"2025-03-21T17:18:05.499Z","etag":null,"topics":["embedded-database","nosql","python","sqlite"],"latest_commit_sha":null,"homepage":"http://lsm-db.readthedocs.org/","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/coleifer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-08-03T06:51:30.000Z","updated_at":"2025-03-14T11:33:43.000Z","dependencies_parsed_at":"2025-01-19T23:14:20.566Z","dependency_job_id":"aeb041f7-b01b-4ee5-9772-94d8d0917d51","html_url":"https://github.com/coleifer/python-lsm-db","commit_stats":null,"previous_names":[],"tags_count":23,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coleifer%2Fpython-lsm-db","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coleifer%2Fpython-lsm-db/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coleifer%2Fpython-lsm-db/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coleifer%2Fpython-lsm-db/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/coleifer","download_url":"https://codeload.github.com/coleifer/python-lsm-db/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246207492,"owners_count":20740723,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["embedded-database","nosql","python","sqlite"],"created_at":"2024-11-06T00:08:22.924Z","updated_at":"2025-03-29T16:02:26.114Z","avatar_url":"https://github.com/coleifer.png","language":"C","readme":"![Python LSM-DB](http://media.charlesleifer.com/blog/photos/lsm.png)\n\nFast Python bindings for [SQLite's LSM key/value store](http://www.sqlite.org/src4/doc/trunk/www/lsmusr.wiki).\nThe LSM storage engine was initially written as part of the experimental\nSQLite4 rewrite (now abandoned). More recently, the LSM source code was moved\ninto the SQLite3 [source tree](https://www.sqlite.org/cgi/src/dir?ci=e148cdad35520e66\u0026name=ext/lsm1)\nand has seen some improvements and fixes. This project uses the LSM code from\nthe SQLite3 source tree.\n\nFeatures:\n\n* Embedded zero-conf database.\n* Keys support in-order traversal using cursors.\n* Transactional (including nested transactions).\n* Single writer/multiple reader MVCC based transactional concurrency model.\n* On-disk database stored in a single file.\n* Data is durable in the face of application or power failure.\n* Thread-safe.\n* Python 2.x and 3.x.\n\nLimitations:\n\n* Not tested on Windoze.\n\nThe source for Python lsm-db is [hosted on GitHub](https://github.com/coleifer/python-lsm-db).\n\nIf you encounter any bugs in the library, please [open an issue](https://github.com/coleifer/python-lsm-db/issues/new),\nincluding a description of the bug and any related traceback.\n\n## Quick-start\n\nBelow is a sample interactive console session designed to show some of the\nbasic features and functionality of the ``lsm-db`` Python library. Also check\nout the [API documentation](https://lsm-db.readthedocs.io/en/latest/api.html).\n\nTo begin, instantiate a `LSM` object, specifying a path to a database file.\n\n```python\n\n\u003e\u003e\u003e from lsm import LSM\n\u003e\u003e\u003e db = LSM('test.ldb')\n```\n\n### Key/Value Features\n\n`lsm-db` is a key/value store, and has a dictionary-like API:\n\n```python\n\n\u003e\u003e\u003e db['foo'] = 'bar'\n\u003e\u003e\u003e print db['foo']\nbar\n\n\u003e\u003e\u003e for i in range(4):\n...     db['k%s' % i] = str(i)\n...\n\n\u003e\u003e\u003e 'k3' in db\nTrue\n\u003e\u003e\u003e 'k4' in db\nFalse\n\n\u003e\u003e\u003e del db['k3']\n\u003e\u003e\u003e db['k3']\nTraceback (most recent call last):\n  File \"\u003cstdin\u003e\", line 1, in \u003cmodule\u003e\n  File \"lsm.pyx\", line 973, in lsm.LSM.__getitem__ (lsm.c:7142)\n  File \"lsm.pyx\", line 777, in lsm.LSM.fetch (lsm.c:5756)\n  File \"lsm.pyx\", line 778, in lsm.LSM.fetch (lsm.c:5679)\n  File \"lsm.pyx\", line 1289, in lsm.Cursor.seek (lsm.c:12122)\n  File \"lsm.pyx\", line 1311, in lsm.Cursor.seek (lsm.c:12008)\nKeyError: 'k3'\n```\n\nBy default when you attempt to look up a key, ``lsm-db`` will search for an\nexact match. You can also search for the closest key, if the specific key you\nare searching for does not exist:\n\n```python\n\n\u003e\u003e\u003e from lsm import SEEK_LE, SEEK_GE\n\u003e\u003e\u003e db['k1xx', SEEK_LE]  # Here we will match \"k1\".\n'1'\n\u003e\u003e\u003e db['k1xx', SEEK_GE]  # Here we will match \"k2\".\n'2'\n```\n\n`LSM` supports other common dictionary methods such as:\n\n* `keys()`\n* `values()`\n* `update()`\n\n### Slices and Iteration\n\nThe database can be iterated through directly, or sliced. When you are slicing\nthe database the start and end keys need not exist -- ``lsm-db`` will find the\nclosest key (details can be found in the [LSM.fetch_range()](https://lsm-db.readthedocs.io/en/latest/api.html#lsm.LSM.fetch_range)\ndocumentation).\n\n```python\n\n\u003e\u003e\u003e [item for item in db]\n[('foo', 'bar'), ('k0', '0'), ('k1', '1'), ('k2', '2')]\n\n\u003e\u003e\u003e db['k0':'k99']\n\u003cgenerator object at 0x7f2ae93072f8\u003e\n\n\u003e\u003e\u003e list(db['k0':'k99'])\n[('k0', '0'), ('k1', '1'), ('k2', '2')]\n```\n\nYou can use open-ended slices. If the lower- or upper-bound is outside the\nrange of keys an empty list is returned.\n\n```python\n\n\u003e\u003e\u003e list(db['k0':])\n[('k0', '0'), ('k1', '1'), ('k2', '2')]\n\n\u003e\u003e\u003e list(db[:'k1'])\n[('foo', 'bar'), ('k0', '0'), ('k1', '1')]\n\n\u003e\u003e\u003e list(db[:'aaa'])\n[]\n```\n\nTo retrieve keys in reverse order, simply use a higher key as the first\nparameter of your slice. If you are retrieving an open-ended slice, you can\nspecify ``True`` as the ``step`` parameter of the slice.\n\n```python\n\n\u003e\u003e\u003e list(db['k1':'aaa'])  # Since 'k1' \u003e 'aaa', keys are retrieved in reverse:\n[('k1', '1'), ('k0', '0'), ('foo', 'bar')]\n\n\u003e\u003e\u003e list(db['k1'::True])  # Open-ended slices specify True for step:\n[('k1', '1'), ('k0', '0'), ('foo', 'bar')]\n```\n\nYou can also **delete** slices of keys, but note that the delete **will not**\ninclude the keys themselves:\n\n```python\n\n\u003e\u003e\u003e del db['k0':'k99']\n\n\u003e\u003e\u003e list(db)  # Note that 'k0' still exists.\n[('foo', 'bar'), ('k0', '0')]\n```\n\n### Cursors\n\nWhile slicing may cover most use-cases, for finer-grained control you can use\ncursors for traversing records.\n\n```python\n\n\u003e\u003e\u003e with db.cursor() as cursor:\n...     for key, value in cursor:\n...         print key, '=\u003e', value\n...\nfoo =\u003e bar\nk0 =\u003e 0\n\n\u003e\u003e\u003e db.update({'k1': '1', 'k2': '2', 'k3': '3'})\n\n\u003e\u003e\u003e with db.cursor() as cursor:\n...     cursor.first()\n...     print cursor.key()\n...     cursor.last()\n...     print cursor.key()\n...     cursor.previous()\n...     print cursor.key()\n...\nfoo\nk3\nk2\n\n\u003e\u003e\u003e with db.cursor() as cursor:\n...     cursor.seek('k0', SEEK_GE)\n...     print list(cursor.fetch_until('k99'))\n...\n[('k0', '0'), ('k1', '1'), ('k2', '2'), ('k3', '3')]\n```\n\nIt is very important to close a cursor when you are through using it. For this\nreason, it is recommended you use the `LSM.cursor()` context-manager, which\nensures the cursor is closed properly.\n\n### Transactions\n\n``lsm-db`` supports nested transactions. The simplest way to use transactions\nis with the `LSM.transaction()` method, which doubles as a context-manager or\ndecorator.\n\n```python\n\n\u003e\u003e\u003e with db.transaction() as txn:\n...     db['k1'] = '1-mod'\n...     with db.transaction() as txn2:\n...         db['k2'] = '2-mod'\n...         txn2.rollback()\n...\nTrue\n\u003e\u003e\u003e print db['k1'], db['k2']\n1-mod 2\n```\n\nYou can commit or roll-back transactions part-way through a wrapped block:\n\n```python\n\n\u003e\u003e\u003e with db.transaction() as txn:\n...    db['k1'] = 'outer txn'\n...    txn.commit()  # The write is preserved.\n...\n...    db['k1'] = 'outer txn-2'\n...    with db.transaction() as txn2:\n...        db['k1'] = 'inner-txn'  # This is commited after the block ends.\n...    print db['k1']  # Prints \"inner-txn\".\n...    txn.rollback()  # Rolls back both the changes from txn2 and the preceding write.\n...    print db['k1']\n...\n1              \u003c- Return value from call to commit().\ninner-txn      \u003c- Printed after end of txn2.\nTrue           \u003c- Return value of call to rollback().\nouter txn      \u003c- Printed after rollback.\n```\n\nIf you like, you can also explicitly call `LSM.begin()`, `LSM.commit()`, and\n`LSM.rollback()`.\n\n```python\n\n\u003e\u003e\u003e db.begin()\n\u003e\u003e\u003e db['foo'] = 'baze'\n\u003e\u003e\u003e print db['foo']\nbaze\n\u003e\u003e\u003e db.rollback()\nTrue\n\u003e\u003e\u003e print db['foo']\nbar\n```\n\n### Reading more\n\nFor more information, check out the project's documentation, hosted at\nreadthedocs:\n\nhttps://lsm-db.readthedocs.io/en/latest/\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcoleifer%2Fpython-lsm-db","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcoleifer%2Fpython-lsm-db","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcoleifer%2Fpython-lsm-db/lists"}