{"id":21040971,"url":"https://github.com/chfoo/grebedb","last_synced_at":"2025-06-12T17:31:50.569Z","repository":{"id":62439761,"uuid":"351562438","full_name":"chfoo/grebedb","owner":"chfoo","description":"Lightweight embedded key-value store/database Rust library","archived":false,"fork":false,"pushed_at":"2021-10-28T00:57:50.000Z","size":364,"stargazers_count":11,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-05-29T21:52:43.020Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chfoo.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-03-25T20:09:06.000Z","updated_at":"2022-12-29T11:32:15.000Z","dependencies_parsed_at":"2022-11-01T21:52:59.690Z","dependency_job_id":null,"html_url":"https://github.com/chfoo/grebedb","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/chfoo/grebedb","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chfoo%2Fgrebedb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chfoo%2Fgrebedb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chfoo%2Fgrebedb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chfoo%2Fgrebedb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chfoo","download_url":"https://codeload.github.com/chfoo/grebedb/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chfoo%2Fgrebedb/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259511990,"owners_count":22869325,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-19T13:49:18.247Z","updated_at":"2025-06-12T17:31:50.549Z","avatar_url":"https://github.com/chfoo.png","language":"Rust","readme":"# GrebeDB\n\nGrebeDB is a Rust library that provides a lightweight embedded key-value store/database backed by files in a virtual file system interface. It is intended for single-process applications that prefer a key-value database-like interface to data instead operating on file formats directly.\n\n[![Crates.io](https://img.shields.io/crates/v/grebedb)](https://crates.io/crates/grebedb) [![docs.rs](https://img.shields.io/docsrs/grebedb)](https://docs.rs/grebedb)\n\n**Note:** While the library is in a very usable state and a decent amount of effort has been made to ensure it is correct and robust, it has not been extensively tested or used in production. Use with caution and make backups regularly of important data.\n\n## Design summary (limitations and guarantees)\n\nSince there are too many key-value stores, the design of the database is immediately described upfront for you to decide whether GrebeDB is fit for your use:\n\n* The database is implemented as a B+ tree with each node saved to a file.\n  * Both keys and values are treated as binary data. Keys with prefixes are not optimized.\n  * Values are stored inline with the leaf nodes.\n  * Lazy deletion is performed.\n* The size of each file is not fixed and can vary significantly depending on the data stored and configuration options.\n* Files are stored using a virtual file system interface. The implementation can be in memory, on a real disk, or your own implementation. Performance and durability is dependent on the file system.\n* Operations such as `get`, `put`, `remove`, and `cursor` are provided. There's no support for traditional transactions, but the `flush` operation provides atomic saving of data. Internal consistency is provided by file copy-on-writes, incrementing revision counters, and atomic file renames.\n* Compression of the file with Zstandard can be used.\n* Concurrency is not supported. No threads are used for background tasks.\n\nFor details about the file format, see [format.md](https://github.com/chfoo/grebedb/blob/main/format.md).\n\n## Getting started\n\nRemember to add the `grebedb` crate dependency to your Cargo.toml.\n\nA GrebeDB database is stored as multiple files in a directory. The following creates a database using the given path and default options:\n\n```rust\nlet options = Options::default();\nlet mut db = Database::open_path(\"path/to/empty/directory/\", options)?;\n```\n\nStoring, retrieving, and deleting keys is as simple as using the `get()`, `put()`, and `remove()` functions:\n\n```rust\ndb.put(\"my_key\", \"hello world\")?;\n\nprintln!(\"The value of my_key is {:?}\", db.get(\"my_key\")?);\n\ndb.remove(\"my_key\")?;\n\nprintln!(\"The value of my_key is now {:?}\", db.get(\"my_key\")?);\n```\n\nTo get all the key-values, use `cursor()`:\n\n```rust\nfor (key, value) in db.cursor() {\n    println!(\"key = {}, value = {}\", key, value);\n}\n```\n\nThe database uses an internal cache and automatically delays writing data to the file system. If you want to ensure all changes has been persisted to the file system at a certain point, use `flush()`. This operation saves the data with atomicity, effectively emulating a transaction, before returning:\n\n```rust\ndb.flush()?;\n```\n\nTip: When inserting a large amount of items, insert keys in sorted order for best performance. Inserting many random keys will be slow as it will require opening, writing, and closing the file that contains the position for it. If you need sorted ID generation, try algorithms based on Twitter Snowflake, MongoDB Object ID, or ULID.\n\nFor more information, check the [examples](https://github.com/chfoo/grebedb/tree/main/src/library/examples) directory and the [API reference on docs.rs](https://docs.rs/grebedb).\n\n### Features\n\nBy default, the features are enabled:\n\n* `compression`: `zstd` crate is enabled for compression\n* `file_locking`: `fslock` is for cross-platform file locking\n* `system`: `getrandom` is a dependency for `uuid`\n\nTo disable them, use `default-features = false` in your Cargo.toml file.\n\n### Tool\n\nFor a command-line tool to provide basic manipulation (such as import \u0026 export for backup) and debugging, see [grebedb-tool](https://github.com/chfoo/grebedb/tree/main/src/tool).\n\n## Contributing\n\nPlease use the GitHub Issues, Pull Requests, and Discussions if you have any problems, bug fixes, or suggestions.\n\n### Roadmap\n\nPossible improvements:\n\n* Better API for buffer arguments. The `\u0026mut Vec\u003cu8\u003e` type might be inflexible.\n* Reduce internal memory allocation and copying.\n* Reduce disk IO with smarter caching.\n* More tests for error cases.\n\n### Not in scope\n\nThe following non-exhaustive features are *not* in scope and likely won't be implemented:\n\n* Additions or modifications of data structures that increase complexity (such as supporting reverse cursor direction).\n* Traditional grouping of operations into transactions with commit and rollback.\n* Threads for performing automatic background work.\n* Async/.await support.\n\nIf you need more features or require better performance, it's likely you need a more powerful database with a Rust binding such as RocksDB or a traditional database like SQLite.\n\n## License\n\nCopyright 2021 Christopher Foo. Licensed under Mozilla Public License 2.0.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchfoo%2Fgrebedb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchfoo%2Fgrebedb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchfoo%2Fgrebedb/lists"}