{"id":27134792,"url":"https://github.com/joedeveloper55/happystore","last_synced_at":"2026-04-19T02:03:12.239Z","repository":{"id":285260148,"uuid":"861016591","full_name":"joedeveloper55/happystore","owner":"joedeveloper55","description":"simple yet feature rich, pure python, embedded key-value database with a tiny api","archived":false,"fork":false,"pushed_at":"2025-03-30T16:12:45.000Z","size":12,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-30T17:24:05.761Z","etag":null,"topics":["database","embedded-database","nosql","object-database","python","python3"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/joedeveloper55.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-21T19:17:49.000Z","updated_at":"2025-03-30T16:12:48.000Z","dependencies_parsed_at":"2025-03-30T17:34:15.751Z","dependency_job_id":null,"html_url":"https://github.com/joedeveloper55/happystore","commit_stats":null,"previous_names":["joedeveloper55/happystore"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joedeveloper55%2Fhappystore","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joedeveloper55%2Fhappystore/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joedeveloper55%2Fhappystore/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joedeveloper55%2Fhappystore/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/joedeveloper55","download_url":"https://codeload.github.com/joedeveloper55/happystore/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247755576,"owners_count":20990626,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","embedded-database","nosql","object-database","python","python3"],"created_at":"2025-04-08T00:59:16.959Z","updated_at":"2026-04-19T02:03:12.198Z","avatar_url":"https://github.com/joedeveloper55.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# happystore\n## Overview\n\nHappyStore is a simple yet feature rich, pure python, embedded key-value database with a tiny api.\n\nIf you need something sort of like shelve from the python standard library, but it's feature set,\nperformance, and robustness just doesn't quite cut it for you, Happystore is the python object \npersistance tool you need.\n\nMajor features include:\n  * can store arbitrary python objects as values. serialization and deserialization are trivially pluggable.\n  * all the expected get, set, delete, and has operations expected of a key-value databse\n  * a means to efficiently query a \"range\" or collection of keys at once, rather than just one at a time\n  * a few more convenience operations for optimizing performance (bulk_get, bulk_set, bulk_delete)\n  * full thread safety and process safety\n  * strictly serializable transactions suppored as a core feature\n  * nestable transactions supported as a core feature\n  * runs in memory or on disk\n  * zero dependencies, pure python\n\nA HappyStore db is a thread-safe, process-safe collection of \"key-value pairs\". Keys are always utf-8\nstrings and values can be arbitrary Python objects.\n\nA HappyStore db can be ran on disk or in memory, just like sqlite if you're familiar with it;\nActually, behind the scenes it is backed by sqlite.\n\nTo connect to a HappyStore, you must explicitly provide a \"serializer object\" to the constructor\nthat controls the serialization and deserialization of python objects to and from bytes (for your\nconvenience, this module comes with some 'batteries included' serializers in the forms of\nPickleSerializer, JsonSerializer and RawSerializer. You can create your own by extending the Serializer abstract\nbase class).\n\n```python\n\u003e\u003e\u003e HappyStore('/tmp/happy_store_db_file.dat', serializer=PickleSerializer())  \n\u003chappystore.HappyStore object at 0x...\u003e\n\n```\nTo run a HappyStore in memory, just use the special \":memory:\" string in place of a filename\n\n```python\n\u003e\u003e\u003e HappyStore(':memory:', serializer=PickleSerializer())  \n\u003chappystore.HappyStore object at 0x...\u003e\n\n```\nso now that you can see how to construct various kinds of HappyStores, let's explore how to\nwork with one\n\n```python\n\u003e\u003e\u003e store = HappyStore(':memory:', serializer=PickleSerializer())\n\n```\nA key value pair is added into the store via the \"set\" method\n\n```python\n\u003e\u003e\u003e store.set('a', 5)\n\n```\nKeys must be strings\n\n```python\n\u003e\u003e\u003e store.set(0, 5)\nTraceback (most recent call last):\n...\nTypeError: key must be a str, not \u003cclass 'int'\u003e\n\n```\nbut values can be any kind of python object (as long as it's \none your serialzer class can turn into bytes)\n\n```python\n\u003e\u003e\u003e store.set('b', {'k': [1, 2]})\n\n```\nTo get a value back out of the store, you use the 'get' method\n\n```python\n\u003e\u003e\u003e store.get('a')\n5\n\n```\nTrying to get a value that doesn't exist results in a LookupError\n\n```python\n\u003e\u003e\u003e store.get('c')\nTraceback (most recent call last):\n...\nLookupError\n\n```\nalternatively, you may check for the existance of a key-value pair with the 'has' method\n\n```python\n\u003e\u003e\u003e store.has('a')\nTrue\n\u003e\u003e\u003e store.has('c')\nFalse\n\n```\nTo remove a key-value pair you use the 'delete' method\n\n```python\n\u003e\u003e\u003e store.delete('b')\nTrue\n\n```\nIt returns true if the key existed and was deleted and False if it never existed\n\nWith the current operations we've shown you (set, get, has, and delete), there's no way to establish\nand search for a \"group\" or \"range\" of key-value pairs. Happystore provides a way for you to do this\nwith its \"query\" method.\n\nfirst we'll set up some keys to search through\n\n```python\n\u003e\u003e\u003e store.set('a', 5)\n\n```\n```python\n\u003e\u003e\u003e store.set('ab', 10)\n\n```\n```python\n\u003e\u003e\u003e store.set('abc', 15)\n\n```\nYou can then search with the 'query' method\n\n```python\n\u003e\u003e\u003e store.query(keyprefix='a') == [('a', 5), ('ab', 10), ('abc', 15)]  # it returns key-value pairs\nTrue\n\n```\nInstead of searching by key prefix, you can also search by range (always inclusive)\n```python\n\u003e\u003e\u003e store.query(start='a', end='ab') == [('a', 5), ('ab', 10)]\nTrue\n\n```\nYou can also iterate through all key-value pairs in the database with the scan method\n\n```python\n\u003e\u003e\u003e list(store.scan())\n[('a', 5), ('ab', 10), ('abc', 15)]\n\n```\nHappystore also provides methods for efficiently performing 'bulk' get, set, and delete operations\n\n```python\n\u003e\u003e\u003e store.bulk_set([('1', 'bacon'), ('2', 'eggs'), ('3', 'cheese')])\n\n```\n```python\n\u003e\u003e\u003e store.bulk_get(['1', '2', '3'])\n['bacon', 'eggs', 'cheese']\n\u003e\u003e\u003e store.bulk_delete(['1', '2', '3'])\n[True, True, True]\n\n```\nFinally, and perhaps the most important feature of all in HappyStore, is it's support for \"transactions\".\n\nThe HappyStore library was designed with strictly serializable transactions down to its very core;\nIn fact, every single operation is implicitly executed in a transaction if not explicitly placed in one.\n\nTo explicitly begin a transaction, you use the \"transaction\" method to get a context manager.\n\n```python\n\u003e\u003e\u003e with store.transaction():  \n...     value = store.get('a')\n...     store.set('a', value + 1)\n... \n\n```\nThe above implements an atomic thread-safe and process-safe increment operation on the \ninteger stored at 'a'.\n\nHappystore Transactions can be explicitly aborted from within by raising an AbortionError,\nand they are implicitly aborted if an exception is thrown inside a trasaction.\n\n```python\n\u003e\u003e\u003e with store.transaction():  \n...     value = store.get('a')\n...     store.delete('a')\n...     if value \u003c 12:\n...         raise AbortionError()\n...\n\u003e\u003e\u003e store.get('a')\n5\n\n```\nThe above conditionally deletes the key-value pair at 'a' if it's value is over 12\n\nIt is worth noting that you are also allowed to nest transactions with the expected\nsemantics\n\nSuch strictly serializable transactions are an incredibly powerful primitive, but caution and discipline is required\nwhen using them. Remember, Happystore only executes one of these transactions at a time, so\nany other threads or processes must 'wait'; A long running transaction has the potential to\nabsoultely destroy performance. Transaction blocks should almost always be kept short, containing\nonly basic, fast maniulations of python objects and HappyStore method calls.\n\nOnce you're done working with a happystore database, you'll want to close it\n\n```python\n\u003e\u003e\u003e store.close()\n\n```\nIf you've read up to this point, you've now more or less learned the entire api. See the api docs\nbelow for some more information.\n\n## API Reference\n### **class** AbortionError\nAn exception to raise inside a transaction to abort\nit and roll it back. A transaction context manager\nwill not re-raise it, unlike other exceptions thrown\ninside it.\n\n\n### **class** SerializationError\nAn exception raised in Serializer implementations\nwhen some object couldn't be serialized.\n\n\n### **class** DeserializationError\nAn exception raised in Serializer implementations\nwhen some object couldn't be deserialized.\n\n\n### **class** Serializer\nAbstract base class used to define your own custom serialization\nand deserialization of objects. For example, the below implements\napplication level encryption of pickles\n\n```python\n\u003e\u003e\u003e import cryptography\n\u003e\u003e\u003e import pickle\n\u003e\u003e\u003e class EncryptedPickleSerializer(Serializer):\n...    def __init__(self, sym_key):\n...        self._cipher = cryptography.fernet.Fernet(sym_key)\n...\n...    def serailaze(self, value):\n...        try:\n...            return self._cipher.encrypt(pickle.dumps(value))\n...        except Exception:\n...            raise SerializationError('couldn\\'t serialize value')\n...\n...    def deserialize(self, bytess):\n...        try:\n...            return pickle.loads(self._cipher.decrypt(value))\n...        except Exception:\n...            raise DeserializationError('couldn\\'t deserialize value')\n...\n\n```\n\n#### *abstract method* serialize(self, value)\nSerialize an object  \n  \nArgs:  \n- value: any kind of object  \n  \nReturns:  \n- bytes: the object serialized to bytes  \n  \nRaises:  \n- SerializationError: if the value couldn't be serialized  \n  \n\n\n#### *abstract method* deserialize(self, bytess)\nDeserialize some bytes  \n  \nArgs:  \n- value (bytes): the bytes to deserialize  \n  \nReturns:  \n- the deserialized object  \n  \nRaises:  \n- DeserializationError: if the value couldn't be deserialized  \n  \n\n\n### **class** PickleSerializer\nA serializer implementation that uses the python pickle\nmodule to serialize and deserialize values. Custom\nPickler and Unpickler classes can be passed to the constructor\nto customize behavior further.\n\n\n#### *method* \\_\\_init\\_\\_(self, pickler_factory=\u003cclass '_pickle.Pickler'\u003e, unpickler_factory=\u003cclass '_pickle.Unpickler'\u003e)\n\n\n#### *method* serialize(self, value)\nSerialize an object with pickler_factory  \n  \nArgs:  \n- value: any kind of object  \n  \nReturns:  \n- bytes: the object serialized to bytes  \n  \nRaises:  \n- SerializationError: if the value couldn't be serialized  \n\n\n#### *method* deserialize(self, bytess)\nDeserialize some bytes with unpickler_factory  \n  \nArgs:  \n- value (bytes): the bytes to deserialize  \n  \nReturns:  \n- the deserialized object  \n  \nRaises:  \n- DeserializationError: if the value couldn't be deserialized  \n  \n\n\n### **class** JsonSerializer\nA serializer implementation that uses the python json module\nto serialize and deserialize values. expects utf-8 encoded json.\n\n\n#### *method* serialize(self, value)\nSerialize an object to utf-8 json  \n  \nArgs:  \n- value: any kind of object  \n  \nReturns:  \n- bytes: the object serialized to bytes  \n  \nRaises:  \n- SerializationError: if the value couldn't be serialized  \n  \n\n\n#### *method* deserialize(self, bytess)\nDeserialize some bytes from utf-8 json  \n  \nArgs:  \n- value (bytes): the bytes to deserialize  \n  \nReturns:  \n- the deserialized object  \n  \nRaises:  \n- DeserializationError: if the value couldn't be deserialized  \n  \n\n\n### **class** RawSerializer\nA serializer implementation that doesn't do any serialization\nor deserialization. Its serialize method just takes bytes\nand returns them as they are, and it's deserialize method does\nthe same.\n\n\n#### *method* serialize(self, value)\nSerialize pure bytes  \n  \nArgs:  \n- value (bytes):  \n  \nReturns:  \n- bytes: the object serialized to bytes  \n  \nRaises:  \n- SerializationError: if the value couldn't be serialized  \n  \n\n\n#### *method* deserialize(self, bytess)\nDeserialize pure bytes  \n  \nArgs:  \n- value (bytes): the bytes to deserialize  \n  \nReturns:  \n- the deserialized object  \n  \nRaises:  \n- DeserializationError: if the value couldn't be deserialized  \n  \n\n\n### **class** HappyStore\n\nThe class for connection to a HappyStore database.\n\n#### *method* \\_\\_init\\_\\_(self, database, serializer, timeout=None)\nmake and connect to a new HappyStore, or connect to an existing one  \n  \nArgs:  \n- database: a path-like object, often just a string denoting the  \nfile the Happystore is located in, or just ':memory:' if it's in memory  \n- serializer (:obj:`Serializer`): A subclass of the abstract Serializer  \nclass, used for turning arbitrary Python objects into bytes and  \nvice versa for persistance. It is possible for this object to raise  \nSerializationError and DeserializationError from its methods.  \n- timeout (float): time in seconds to wait on an operation until  \nraising an exception from sqlite. passed through to the underlying  \nsqlite3 connection  \n  \nReturns:  \n- A new HappyStore object for interacting with the database  \n\n\n#### *method* close(self)\nclose the connection to the HappyStore database\n\n#### *method* \\_\\_enter\\_\\_(self)\n\n\n#### *method* \\_\\_exit\\_\\_(self, exc_type, exc_val, exc_tb)\n\n\n#### *method* get(self, key)\ntry to get the key-value pair by key  \n  \nArgs:  \n- key (str):  \n  \nReturns:  \n- object: The value of the key value pair  \n  \nRaises:  \n- LookupError: If the key-value pair isn't in the HappyStore  \n- DeserializationError: if the value couldn't be deserialized  \n\n\n#### *method* bulk_get(self, keys)\nget a bunch of key-value pairs at once  \n  \nArgs:  \n- keys: a list of the keys to get  \n  \nReturns:  \n- a list of values and/or LookupErrors  \n\n\n#### *method* has(self, key)\ntest if the key is in the HappyStore  \n  \nArgs:  \n- key (str):  \n  \nReturns:  \n- bool: The value of the key value pair  \n\n\n#### *method* set(self, key, value)\ntry to set the key-value pair by key  \n  \nArgs:  \n- key (str):  \n- value (object):  \n  \nRaises:  \n- SerializationError: if the value couldn't be serialized  \n\n\n#### *method* bulk_set(self, key_value_pairs)\nset a bunch of key-value pairs at once  \n  \nArgs:  \n- key_value_pairs: a list of tuples (first item is key, second is value)  \n  \nRaises:  \n- SerializationError: if the value couldn't be serialized  \n\n\n#### *method* delete(self, key)\nremove the key-value pair from the HappyStore  \n  \nArgs:  \n- key (str):  \n  \nReturns:  \n- bool: True if the key found and deleted, False if it didn't exist  \n\n\n#### *method* bulk_delete(self, keys)\nremove a bunch of key-value pairs at once  \n  \nArgs:  \n- keys: a list of the keys to remove  \n  \nReturns:  \n- a list of boolean values for each key removed  \n\n\n#### *method* query(self, keyprefix=None, start=None, end=None, limit=None, reverse=False)\nsearch for a range of key-value pairs in the happystore.  \nIt is efficient and will locate your range in approximately  \nO(log(n)) time complexity.  \n  \nFor the arguments, you must supply only either keyprefix,  \nor start and end. You can't supply all three.  \n  \nArgs:  \n- keyprefix (str): search for all keys with prefix  \n- start (str): search for all keys lexicogrphically greater than or equal to start  \n- end (str): search for all keys lexicogrphically less than or equal to end  \n- limit (int): only return at most the specified number of key-value pairs  \n- reverse (bool): if True, search the range in reverse order (max to min).  \ndefaults to False (min to max)  \n  \nReturns:  \n- list: a list of tuples containing the found key value pairs     \n\n\n#### *method* scan(self, pagesize=100)\niterate through key-value pairs in the happystore, one page  \nat a time. pagesize can be adjusted to fine tune the performance.  \nlarger page sizes consume more memory but perform less io.  \n  \nArgs:  \n- pagesize(int): number of key-value pairs per page  \n  \nReturns:  \n- an iterator of of key-value pairs  \n\n\n#### *method* transaction(self)\nreturns a context manager for executing operations  \ninside a transaction. All operations are strictly  \nserializable.  \n  \nAn AbortionError can be raised inside a transaction  \nblock to explicitly rollback the transaction. It is  \nnever re-raised from the context manger.  \n  \nAny other exceptions raised inside the transaction  \nblock will implicitly rollback the transaction and  \nbe re-raised.  \n  \nThe changes are comited at the end of the block if no  \nexceptions happened.  \n  \nTransactions can be nested.  \n  \nReturns:  \n- A context manager for wrapping a transaction  \n\n\n\n\n## authors/contributors\n * Joeph P McAnulty\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjoedeveloper55%2Fhappystore","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjoedeveloper55%2Fhappystore","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjoedeveloper55%2Fhappystore/lists"}