{"id":13632252,"url":"https://github.com/fjall-rs/fjall","last_synced_at":"2026-02-27T17:16:54.438Z","repository":{"id":210972579,"uuid":"719588139","full_name":"fjall-rs/fjall","owner":"fjall-rs","description":"🗻 Log-structured, embeddable key-value storage engine written in Rust","archived":false,"fork":false,"pushed_at":"2026-02-21T20:40:03.000Z","size":4061,"stargazers_count":1900,"open_issues_count":29,"forks_count":77,"subscribers_count":14,"default_branch":"main","last_synced_at":"2026-02-21T23:06:49.450Z","etag":null,"topics":["database","embeddable","embeddable-database","embedded-database","embedded-kv","key-value-database","key-value-storage","key-value-store","kv","kv-database","kv-storage","kv-store","log-structured","log-structured-merge-tree","lsm","lsm-tree","lsmt","rust","rust-lang","storage-engine"],"latest_commit_sha":null,"homepage":"https://fjall-rs.github.io","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fjall-rs.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null},"funding":{"github":"fjall-rs"}},"created_at":"2023-11-16T13:41:50.000Z","updated_at":"2026-02-21T16:46:42.000Z","dependencies_parsed_at":"2025-11-19T23:02:49.770Z","dependency_job_id":null,"html_url":"https://github.com/fjall-rs/fjall","commit_stats":{"total_commits":677,"total_committers":3,"mean_commits":"225.66666666666666","dds":"0.10930576070901032","last_synced_commit":"854eb199ee349086db1f6842c2fa4302b763500c"},"previous_names":["marvin-j97/lsm-tree","fjall-rs/fjall"],"tags_count":92,"template":false,"template_full_name":null,"purl":"pkg:github/fjall-rs/fjall","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fjall-rs%2Ffjall","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fjall-rs%2Ffjall/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fjall-rs%2Ffjall/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fjall-rs%2Ffjall/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fjall-rs","download_url":"https://codeload.github.com/fjall-rs/fjall/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fjall-rs%2Ffjall/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29901168,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-27T14:46:13.553Z","status":"ssl_error","status_checked_at":"2026-02-27T14:46:10.522Z","response_time":57,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","embeddable","embeddable-database","embedded-database","embedded-kv","key-value-database","key-value-storage","key-value-store","kv","kv-database","kv-storage","kv-store","log-structured","log-structured-merge-tree","lsm","lsm-tree","lsmt","rust","rust-lang","storage-engine"],"created_at":"2024-08-01T22:02:57.836Z","updated_at":"2026-02-27T17:16:54.433Z","avatar_url":"https://github.com/fjall-rs.png","language":"Rust","funding_links":["https://github.com/sponsors/fjall-rs"],"categories":["Rust","\u003ca name=\"Rust\"\u003e\u003c/a\u003eRust"],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"/kawaii.png\" height=\"200\"\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/fjall-rs/fjall/actions/workflows/test.yml\"\u003e\n    \u003cimg src=\"https://github.com/fjall-rs/fjall/actions/workflows/test.yml/badge.svg\" alt=\"CI\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://docs.rs/fjall\"\u003e\n    \u003cimg src=\"https://img.shields.io/docsrs/fjall?color=green\" alt=\"docs.rs\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://crates.io/crates/fjall\"\u003e\n    \u003cimg src=\"https://img.shields.io/crates/v/fjall?color=blue\" alt=\"Crates.io\" /\u003e\n  \u003c/a\u003e\n  \u003cimg src=\"https://img.shields.io/badge/MSRV-1.91.0-blue\" alt=\"MSRV\" /\u003e\n  \u003ca href=\"https://deps.rs/repo/github/fjall-rs/fjall\"\u003e\n    \u003cimg src=\"https://deps.rs/repo/github/fjall-rs/fjall/status.svg\" alt=\"dependency status\" /\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://discord.com/invite/HvYGp4NFFk\"\u003e\n    \u003cimg src=\"https://img.shields.io/discord/1240426554111164486\" alt=\"Discord\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://bsky.app/profile/fjallrs.bsky.social\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/bluesky-blue\" alt=\"Bluesky\" /\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n*Fjall* _(Nordic: \"Mountain\")_ is a log-structured, embeddable key-value storage engine written in Rust.\nIt features:\n\n- A thread-safe BTreeMap-like API\n- 100% safe \u0026 stable Rust\n- LSM-tree-based storage similar to `RocksDB`\n- Range \u0026 prefix searching with forward and reverse iteration\n- Multiple keyspaces (a.k.a. column families) with cross-keyspace atomic semantics\n- Built-in compression (default = `LZ4`)\n- Serializable transactions (optional)\n- Key-value separation for large blob use cases (optional)\n- Automatic background maintenance\n\nIt is not:\n\n- A standalone database server\n- A relational or wide-column database: it has no built-in notion of columns or query language\n\n## Sponsors\n\n\u003ca href=\"https://sqlsync.dev\"\u003e\n  \u003cpicture\u003e\n    \u003csource width=\"240\" alt=\"Orbitinghail\" media=\"(prefers-color-scheme: light)\" srcset=\"https://raw.githubusercontent.com/fjall-rs/fjall-rs.github.io/d22fcb1e6966ce08327ea3bf6cf2ea86a840b071/public/logos/orbitinghail.svg\" /\u003e\n    \u003csource width=\"240\" alt=\"Orbitinghail\" media=\"(prefers-color-scheme: dark)\" srcset=\"https://raw.githubusercontent.com/fjall-rs/fjall-rs.github.io/d22fcb1e6966ce08327ea3bf6cf2ea86a840b071/public/logos/orbitinghail_dark.svg\" /\u003e\n    \u003cimg width=\"240\" alt=\"Orbitinghail\" src=\"https://raw.githubusercontent.com/fjall-rs/fjall-rs.github.io/d22fcb1e6966ce08327ea3bf6cf2ea86a840b071/public/logos/orbitinghail_dark.svg\" /\u003e\n  \u003c/picture\u003e\n\u003c/a\u003e\n\n## Basic usage\n\n```bash\ncargo add fjall\n```\n\n```rust\nuse fjall::{Database, KeyspaceCreateOptions, PersistMode};\n\nfn main() -\u003e fjall::Result\u003c()\u003e {\n    // A database may contain multiple keyspaces\n    // You should probably only use a single database for your application\n    let db = Database::builder(path).open()?;\n    // TxDatabase::builder for transactional semantics\n\n    // Each keyspace is its own physical LSM-tree, and thus isolated from other keyspaces\n    let items = db.keyspace(\"my_items\", KeyspaceCreateOptions::default)?;\n\n    // Write some data\n    items.insert(\"a\", \"hello\")?;\n\n    // And retrieve it\n    let bytes = items.get(\"a\")?;\n\n    // Or remove it again\n    items.remove(\"a\")?;\n\n    // Search by prefix\n    for kv in items.prefix(\"prefix\") {\n        // ...\n    }\n\n    // Search by range\n    for kv in items.range(\"a\"..=\"z\") {\n        // ...\n    }\n\n    // Iterators implement DoubleEndedIterator, so you can search backwards, too!\n    for kv in items.prefix(\"prefix\").rev() {\n        // ...\n    }\n\n    // Sync the journal to disk to make sure data is definitely durable\n    // When the database is dropped, it will try to persist with `PersistMode::SyncAll` automatically\n    db.persist(PersistMode::SyncAll)\n}\n```\n\n\u003e [!TIP]\n\u003e Like any typical key-value store, keys are stored in lexicographic order.\n\u003e If you are storing integer keys (e.g. timeseries data), you should use the big endian form to have predictable ordering.\n\n## Durability\n\nTo support different kinds of workloads, Fjall is agnostic about the type of durability\nyour application needs.\nAfter writing data (`insert`, `remove` or committing a write batch/transaction), you can choose to call [`Database::persist`](https://docs.rs/fjall/latest/fjall/struct.Database.html#method.persist) which takes a [`PersistMode`](https://docs.rs/fjall/latest/fjall/enum.PersistMode.html) parameter.\nBy default, any operation will flush to OS buffers, but **not** to disk.\nThis matches RocksDB's default durability.\nAlso, when dropped, the database will try to persist the journal *to disk* synchronously.\n\n## Multithreading, Async and Multiprocess\n\n\u003e [!WARNING]\n\u003e A single database may **not** be loaded in parallel from separate *processes*.\n\nFjall is internally synchronized for multi-*threaded* access, so you can clone around the `Database` and `Keyspace`s as needed, without needing to lock yourself.\n\nFor an async example, see the [`tokio`](https://github.com/fjall-rs/fjall/tree/main/examples/tokio) example.\n\n## Memory usage\n\nGenerally, memory for loaded data, indexes etc. is managed on a per-block basis, and capped by the block cache capacity.\nNote that this also applies to returned values: When you hold a `Slice`, it keeps the backing buffer alive (which may be a block).\nIf you know that you are going to keep a value around for a long time, you may want to copy it out into a new `Vec\u003cu8\u003e`, `Box\u003c[u8]\u003e`, `Arc\u003c[u8]\u003e` or new `Slice` (using `Slice::new`).\n\n\u003e [!NOTE]\n\u003e It is recommended to configure the block cache capacity to be ~20-25% of the available memory - or more **if** the data set fits _fully_ into memory.\n\nAdditionally, orthogonally to the block cache, each `Keyspace` has its own write buffer ([\"Memtable\"](https://docs.rs/fjall/latest/fjall/struct.KeyspaceCreateOptions.html#method.max_memtable_size)) which is the unit of data flushed back into the \"proper\" index structure.\n\n## Error handling\n\nFjall returns an [error enum](https://docs.rs/fjall/latest/fjall/enum.Error.html), however these variants are mostly used for debugging and tracing purposes, so your application is not expected to handle specific errors.\n\nIt's best to let the application crash and restart, which is the [safest way to recover from transient I/O errors](https://ramalagappan.github.io/pdfs/papers/cuttlefs.pdf).\n\n## Transactional modes\n\nThe backing store (`lsm-tree`) is a MVCC key-value store, allowing repeatable snapshot reads.\nHowever this isolation level can not do read-modify-write operations without the chance of lost updates.\nAlso, `WriteBatch` does not allow reading the intermediary state back as you would expect from a proper transaction.\nFor that reason, if you need transactional semantics, you need to use one of the transactional database implementation (`OptimisticTxDatabase` or `SingleWriterTxDatabase`).\n\nTL;DR: Fjall supports both transactional and non-transactional workloads.\nChances are you want to use a transactional database, unless you know your workload does not need serializable transaction semantics.\n\n### Single writer\n\nOpens a transactional database for single-writer (serialized) transactions.\nSingle writer means only a single **write** transaction can run at a time.\nThis is trivially serializable because it _literally_ serializes write transactions.\n\n### Optimistic\n\nOpens a transactional database for multi-writer, serializable transactions.\nConflict checking is done using optimistic concurrency control, meaning transactions can conflict and may have to be rerun.\n\n## Feature flags\n\n### lz4\n\nAllows using `LZ4` compression, powered by [`lz4_flex`](https://github.com/PSeitz/lz4_flex).\n\n*Enabled by default.*\n\n### bytes_1\n\nUses [`bytes`](https://github.com/tokio-rs/bytes) 1.x as the underlying `Slice` type.\nOtherwise, [`byteview`](https://github.com/fjall-rs/byteview) is used instead.\n\n*Disabled by default.*\n\n## Stable disk format\n\nFuture breaking changes will result in a major version bump and a migration path.\n\nFor the underlying LSM-tree implementation, see: \u003chttps://crates.io/crates/lsm-tree\u003e.\n\n## Examples\n\n[See here](https://github.com/fjall-rs/fjall/tree/main/examples) for practical examples.\n\n## Contributing\n\nHow can you help?\n\n- [Ask a question](https://github.com/fjall-rs/fjall/discussions/new?category=q-a)\n  - or join the Discord server: [https://discord.com/invite/HvYGp4NFFk](https://discord.com/invite/HvYGp4NFFk)\n- [Post benchmarks and things you created](https://github.com/fjall-rs/fjall/discussions/new?category=show-and-tell)\n- [Open a PR](https://github.com/fjall-rs/fjall/compare),\n  - [See open issues to pick up here](https://github.com/search?q=org%3Afjall-rs+label%3A%22help+wanted%22+state%3Aopen+\u0026type=issues)\n- [Open an issue](https://github.com/fjall-rs/fjall/issues/new) (bug report, weirdness)\n\n## License\n\nAll source code is licensed under MIT OR Apache-2.0.\n\nAll contributions are to be licensed as MIT OR Apache-2.0.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffjall-rs%2Ffjall","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffjall-rs%2Ffjall","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffjall-rs%2Ffjall/lists"}