{"id":13936583,"url":"https://github.com/Miserlou/NoDB","last_synced_at":"2025-07-19T22:30:54.213Z","repository":{"id":57446511,"uuid":"87740586","full_name":"Miserlou/NoDB","owner":"Miserlou","description":"NoDB isn't a database.. but it sort of looks like one.","archived":false,"fork":false,"pushed_at":"2020-05-05T17:35:46.000Z","size":59,"stargazers_count":386,"open_issues_count":12,"forks_count":45,"subscribers_count":16,"default_branch":"master","last_synced_at":"2025-06-29T17:48:57.337Z","etag":null,"topics":["aws","json","lambda","nodb","nosql","python","s3","serverless","zappa"],"latest_commit_sha":null,"homepage":"https://blog.zappa.io/posts/introducing-nodb-pythonic-data-store-s3","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Miserlou.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-04-09T21:35:55.000Z","updated_at":"2025-05-31T00:03:27.000Z","dependencies_parsed_at":"2022-09-02T22:11:42.341Z","dependency_job_id":null,"html_url":"https://github.com/Miserlou/NoDB","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/Miserlou/NoDB","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Miserlou%2FNoDB","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Miserlou%2FNoDB/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Miserlou%2FNoDB/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Miserlou%2FNoDB/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Miserlou","download_url":"https://codeload.github.com/Miserlou/NoDB/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Miserlou%2FNoDB/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266030815,"owners_count":23866621,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","json","lambda","nodb","nosql","python","s3","serverless","zappa"],"created_at":"2024-08-07T23:02:48.622Z","updated_at":"2025-07-19T22:30:53.886Z","avatar_url":"https://github.com/Miserlou.png","language":"Python","readme":"\u003cimg src=\"http://i.imgur.com/ZymFZd8.jpg\" width=\"400\"/\u003e\n\n# NoDB\n\n[![Build Status](https://travis-ci.org/Miserlou/NoDB.svg)](https://travis-ci.org/Miserlou/NoDB)\n[![Coverage](https://img.shields.io/coveralls/Miserlou/NoDB.svg)](https://coveralls.io/github/Miserlou/NoDB)\n[![PyPI](https://img.shields.io/pypi/v/NoDB.svg)](https://pypi.python.org/pypi/nodb)\n[![Slack](https://img.shields.io/badge/chat-slack-ff69b4.svg)](https://slack.zappa.io/)\n[![Gun.io](https://img.shields.io/badge/made%20by-gun.io-blue.svg)](https://gun.io/)\n[![Patreon](https://img.shields.io/badge/support-patreon-brightgreen.svg)](https://patreon.com/zappa)\n\n_NoDB isn't a database.. but it sort of looks like one!_\n\n**NoDB** an incredibly simple, Pythonic object store based on Amazon's S3 static file storage.\n\nIt's useful for **prototyping**, **casual hacking**, and (maybe) even low-traffic **server-less backends** for **[Zappa](https://github.com/Miserlou/Zappa) apps**!\n\n## Features\n\n* Schema-less!\n* Server-less!\n* Uses S3 as a datastore.\n* Loads to native Python objects with `cPickle`\n* Can use JSON as a serialization format for untrusted data\n* Local filestore based caching\n* Cheap(ish)!\n* Fast(ish)! (Especially from Lambda)\n\n## Performance\n\nInitial load test with [Goad](https://goad.io/) of 10,000 requests (500 concurrent) with a write and subsequent read of the same index showed an average time of 400ms. This should be more than acceptable for many applications, even those which don't have sparse data, although that is preferred.\n\n## Installation\n\n**NoDB** can be installed easily via `pip`, like so:\n\n```\n$ pip install nodb\n```\n\n## Warning!\n**NoDB** is **insecure by default**! Do not use it for untrusted data before setting `serializer` to `\"json\"`!\n\n## Usage\n\n**NoDB** is super easy to use!\n\nYou simply make a NoDB object, point it to your bucket and tell it what field you want to index on.\n\n```python\nfrom nodb import NoDB\n\nnodb = NoDB(\"my-s3-bucket\")\nnodb.index = \"name\"\n```\n\nAfter that, you can save and load literally anything you want, whenever you want!\n\n```python\n# Save an object!\nuser = {\"name\": \"Jeff\", \"age\": 19}\nnodb.save(user) # True\n\n# Load our object!\nuser = nodb.load(\"Jeff\")\nprint(user['age']) # 19\n\n# Delete our object\nnodb.delete(\"Jeff\") # True\n```\n\nBy default, you can save and load any Python object.\n\nHere's the same example, but with a class. Note the import and configuration is the same!\n\n```python\nclass User(object):\n    name = None\n    age = None\n    \n    def print_name(self):\n        print(\"Hi, I'm \" + self.name + \"!\")\n    \n    def __repr__(self):\n        \"\"\" show a human readable representation of this class \"\"\"\n        return \"\u003c%s: %s (%s)\u003e\" % (self.__class__.__name__, self.name, self.age)\n\nnew_user = User()\nnew_user.name = \"Jeff\"\nnew_user.age = 19\nnodb.save(new_user) \n# True\n\njeff = nodb.load(\"Jeff\")\njeff.print_name() \n# Hi, I'm Jeff!\n```\n\nYou can return a list of all objects using the .all() method.\n\nHere's an example following from the code above, adding some extra users to the database and then listing all. \n\n```python\nnewer_user = User()\nnewer_user.name = \"Ben\"\nnewer_user.age = 38\nnodb.save(newer_user)\n# True\n\nnewest_user = User()\nnewest_user.name = \"Thea\"\nnewest_user.age = 33\nnodb.save(newest_user)\n# True\n\nnodb.all()\n# [\u003cUser: Jeff (19)\u003e, \u003cUser: Ben (38)\u003e, \u003cUser: Thea (33)\u003e]\n```\n\n## Advanced Usage\n\n### Different Serializers\n\nTo use a safer, non-Pickle serializer, just set JSON as your serializer:\n\n```python\nnodb = NoDB()\nnodb.serializer = \"json\"\n```\n\nNote that for this to work, your object must be JSON-serializable.\n\n### Object Metadata\n\nYou can get metainfo (datetime and UUID) for a given object by passing `metainfo=True` to `load`, like so:\n\n```python\n# Load our object and metainfo!\nuser, datetime, uuid = nodb.load(\"Jeff\", metainfo=True)\n```\n\nYou can also pass in a `default` argument for non-existent values.\n\n```python\nuser = nodb.load(\"Jeff\", default={}) # {}\n```\n\n### Human Readable Indexes\n\nBy default, the indexes are hashed. If you want to be able to debug through the AWS console, set `human_readable_indexes` to True:\n\n```python\nnodb.human_readable_indexes = True\n```\n\n### Caching\n\nYou can enable local file caching, which will store previously retrieved values in the local rather than remote filestore.\n\n```python\nnodb.cache = True\n```\n\n### AWS settings override\n\nYou can override your AWS Profile information or boto3 session by passing either as a initial keyword argument.\n\n```python\nnodb = NoDB(profile_name='my_aws_development_profile')\n# or supply the session\nsession = boto3.Session(\n    aws_access_key_id=ACCESS_KEY,\n    aws_secret_access_key=SECRET_KEY,\n    aws_session_token=SESSION_TOKEN,\n)\nnodb = NoDB(session=session)\n```\n\n\n\n## TODO (Maybe?)\n\n* **Tests** with Placebo\n* Local file storage\n* Quering ranges (numberic IDs only), etc.\n* Different serializers\n* Custom serializers\n* Multiple indexes\n* Compression\n* Bucket management\n* Pseudo-locking\n* Performance/load testing\n\n## Related Projects\n\n* [Zappa](https://github.com/Miserlou/Zappa) - Python's server-less framework!\n* [K.E.V.](https://github.com/capless/kev) - a Python ORM for key-value stores based on Redis, S3, and a S3/Redis hybrid backend.\n* [s3sqlite](https://github.com/Miserlou/zappa-django-utils#using-an-s3-backed-database-engine) - An S3-backed database engine for Django\n\n## Contributing\n\nThis project is still young, so there is still plenty to be done. Contributions are more than welcome!\n\nPlease file tickets for discussion before submitting patches. Pull requests should target `master` and should leave NoDB in a \"shippable\" state if merged.\n\nIf you are adding a non-trivial amount of new code, please include a functioning test in your PR. For AWS calls, we use the `placebo` library, which you can learn to use [in their README](https://github.com/garnaat/placebo#usage-as-a-decorator). The test suite will be run by [Travis CI](https://travis-ci.org/Miserlou/NoDB) once you open a pull request.\n\nPlease include the GitHub issue or pull request URL that has discussion related to your changes as a comment in the code ([example](https://github.com/Miserlou/Zappa/blob/fae2925431b820eaedf088a632022e4120a29f89/zappa/zappa.py#L241-L243)). This greatly helps for project maintainability, as it allows us to trace back use cases and explain decision making.\n\n## License\n\n(C) Rich Jones 2017, MIT License.\n\n\u003cbr /\u003e\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://gun.io\"\u003e\u003cimg src=\"http://i.imgur.com/M7wJipR.png\" alt=\"Made by Gun.io\"/\u003e\u003c/a\u003e\n\u003c/p\u003e\n","funding_links":["https://patreon.com/zappa"],"categories":["Python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FMiserlou%2FNoDB","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FMiserlou%2FNoDB","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FMiserlou%2FNoDB/lists"}