{"id":13631887,"url":"https://github.com/khonsulabs/nebari","last_synced_at":"2025-06-18T03:11:04.967Z","repository":{"id":39341668,"uuid":"410309115","full_name":"khonsulabs/nebari","owner":"khonsulabs","description":"A pure Rust database implementation using an append-only B-Tree file format.","archived":false,"fork":false,"pushed_at":"2023-10-11T17:52:51.000Z","size":386482,"stargazers_count":268,"open_issues_count":26,"forks_count":9,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-06-06T21:41:51.440Z","etag":null,"topics":["b-tree","database","rust"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/khonsulabs.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE-APACHE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null},"funding":{"github":["ecton"]}},"created_at":"2021-09-25T15:30:55.000Z","updated_at":"2025-03-31T14:03:59.000Z","dependencies_parsed_at":"2023-10-11T22:35:39.818Z","dependency_job_id":null,"html_url":"https://github.com/khonsulabs/nebari","commit_stats":{"total_commits":176,"total_committers":4,"mean_commits":44.0,"dds":0.02840909090909094,"last_synced_commit":"a94853fdd02dddb394d8b79faaf0df292011073f"},"previous_names":[],"tags_count":15,"template":false,"template_full_name":null,"purl":"pkg:github/khonsulabs/nebari","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/khonsulabs%2Fnebari","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/khonsulabs%2Fnebari/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/khonsulabs%2Fnebari/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/khonsulabs%2Fnebari/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/khonsulabs","download_url":"https://codeload.github.com/khonsulabs/nebari/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/khonsulabs%2Fnebari/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260477932,"owners_count":23015066,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["b-tree","database","rust"],"created_at":"2024-08-01T22:02:42.565Z","updated_at":"2025-06-18T03:10:59.950Z","avatar_url":"https://github.com/khonsulabs.png","language":"Rust","funding_links":["https://github.com/sponsors/ecton"],"categories":["Rust"],"sub_categories":[],"readme":"# Nebari\n\n![nebari forbids unsafe code](https://img.shields.io/badge/unsafe-forbid-success)\n![nebari is considered alpha](https://img.shields.io/badge/status-alpha-orange)\n[![crate version](https://img.shields.io/crates/v/nebari.svg)](https://crates.io/crates/nebari)\n[![Live Build Status](https://img.shields.io/github/actions/workflow/status/khonsulabs/nebari/tests.yml?branch=main)](https://github.com/khonsulabs/nebari/actions?query=workflow:Tests)\n[![HTML Coverage Report for `main` branch](https://khonsulabs.github.io/nebari/coverage/badge.svg)](https://nebari.bonsaidb.io/coverage/)\n[![Documentation for `main` branch](https://img.shields.io/badge/docs-main-informational)](https://nebari.bonsaidb.io/main/nebari/)\n\n\u003e nebari - noun - the surface roots that flare out from the base of a bonsai tree\n\nThis crate provides the `Roots` type, which is the transactional storage layer\nfor [`BonsaiDb`][bonsaidb]. It is loosely inspired by\n[`Couchstore`](https://github.com/couchbase/couchstore).\n\nThis crate blocks the current thread when accessing the filesystem. If you are looking for an async-ready database, [BonsaiDb][bonsaidb] is our vision of an async-aware database built atop Nebari.\n\nThis crate is alpha. While its format is considered stable, there may be bugs\nthat could lead to data loss. Please have a good backup strategy while using\nthis crate.\n\n## Examples\n\nInserting a key-value pair in an on-disk tree with full revision history:\n\n```rust\nuse nebari::{\n    tree::{Root, Versioned},\n    Config,\n};\n\nlet database_folder = tempfile::tempdir().unwrap();\nlet roots = Config::default_for(database_folder.path())\n    .open()\n    .unwrap();\nlet tree = roots.tree(Versioned::tree(\"a-tree\")).unwrap();\ntree.set(\"hello\", \"world\").unwrap();\n```\n\nFor more examples, check out [`nebari/examples/`](https://github.com/khonsulabs/nebari/tree/main/nebari/examples/).\n\n## Features\n\nNebari exposes multiple levels of functionality. The lowest level functionality\nis the\n[`TreeFile`](https://nebari.bonsaidb.io/main/nebari/tree/struct.TreeFile.html).\nA `TreeFile` is a key-value store that uses an append-only file format for its\nimplementation.\n\nUsing `TreeFile`s and a transaction log,\n[`Roots`](https://nebari.bonsaidb.io/main/nebari/struct.Roots.html) enables\nACID-compliant, multi-tree transactions.\n\nEach tree supports:\n\n- **Key-value storage**: Keys can be any arbitrary byte sequence up to 65,535\n  bytes long. For efficiency, keys should be kept to smaller lengths. Values can\n  be up to 4 gigabytes (2^32 bytes) in size.\n- **Flexible query options**: Fetch records one key at a time, multiple keys at\n  once, or ranges of keys.\n- **Powerful multi-key operations**: Internally, all functions that alter the\n  data in a tree use\n  [`TreeFile::modify()`](https://nebari.bonsaidb.io/main/nebari/tree/struct.TreeFile.html#method.modify)\n  which allows operating on one or more keys and performing [various\n  operations](https://nebari.bonsaidb.io/main/nebari/tree/enum.Operation.html).\n- **Pluggable low-level modifications**: The [`Vault`\n  trait](https://nebari.bonsaidb.io/main/nebari/trait.Vault.html) allows you to\n  bring your own encryption, compression, or other functionality to this format.\n  Each independently-addressible chunk of data that is written to the file\n  passes through the vault.\n- **Optional full revision history**. If you don't want to lose old revisions of\n  data, you can use a\n  [`VersionedTreeRoot`](https://nebari.bonsaidb.io/main/nebari/tree/struct.VersionedTreeRoot.html)\n  to store information that allows scanning old revision information. Or, if you\n  want to avoid the extra IO, use the\n  [`UnversionedTreeRoot`](https://nebari.bonsaidb.io/main/nebari/tree/struct.UnversionedTreeRoot.html)\n  which only stores the information needed to retrieve the latest data in the\n  file.\n- **[ACID](https://en.wikipedia.org/wiki/ACID)-compliance**:\n  - **Atomicity**: Every operation on a `TreeFile` is done atomically.\n    [`Operation::CompareSwap`](https://nebari.bonsaidb.io/main/nebari/tree/enum.Operation.html#variant.CompareSwap)\n    can be used to perform atomic operations that require evaluating the\n    currently stored value.\n  - **Consistency**: Atomic locking operations are used when publishing a new\n    transaction state. This ensures that readers can never operate on a partially\n    updated state.\n  - **Isolation**: Currently, each tree can only be accessed exclusively within\n    a transaction. This means that if two transactions request the same tree,\n    one will execute and complete before the second is allowed access to the\n    tree. This strategy could be modified in the future to allow for more\n    flexibility.\n  - **Durability**: The append-only file format is designed to only allow\n    reading data that has been fully flushed to disk. Any writes that were\n    interrupted will be truncated from the end of the file.\n\n    Transaction IDs are recorded in the tree headers. When restoring from disk,\n    the transaction IDs are verified with the transaction log. Because of the\n    append-only format, if we encounter a transaction that wasn't recorded, we\n    can continue scanning the file to recover the previous state. We do this\n    until we find a successfluly commited transaction.\n\n    This process is much simpler than most database implementations due to the\n    simple guarantees that append-only formats provide.\n\n### Why use an append-only file format?\n\n[@ecton](https://github.com/ecton) wasn't a database engineer before starting\nthis project, and depending on your viewpoint may still not be considered a\ndatabase engineer. Implementing ACID-compliance is not something that should be\nattempted lightly.\n\nCreating ACID-compliance with append-only formats is much easier to achieve,\nhowever, as long as you can guarantee two things:\n\n- When opening a previously existing file, can you identify where the last valid\n  write occurred?\n- When writing the file, do not report that a transaction has succeeded until\n  the file is fully flushed to disk.\n\nThe B-Tree implementation in Nebari is designed to offer those exact guarantees.\n\nThe major downside of append-only formats is that deleted data isn't cleaned up\nuntil a maintenance process occurs: compaction. This process rewrites the file's\ncontents, skipping over entries that are no longer alive. This process can\nhappen without blocking the file from being operated on, but it does\nintroduce IO overhead during the operation.\n\nNebari provides APIs that perform compaction, but currently delegates scheduling\nand automation to consumers of this library.\n\n[bonsaidb]: https://bonsaidb.io/\n\n## Open-source Licenses\n\nThis project, like all projects from [Khonsu Labs](https://khonsulabs.com/), are\nopen-source. This repository is available under the [MIT License](./LICENSE-MIT)\nor the [Apache License 2.0](./LICENSE-APACHE).\n\nTo learn more about contributing, please see [CONTRIBUTING.md](./CONTRIBUTING.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkhonsulabs%2Fnebari","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkhonsulabs%2Fnebari","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkhonsulabs%2Fnebari/lists"}