https://github.com/fjall-rs/fjall

LSM-based embeddable key-value storage engine written in safe Rust
https://github.com/fjall-rs/fjall

database embeddable embeddable-database embedded-database embedded-kv key-value-database key-value-store kv kv-database kv-storage kv-store log-structured log-structured-merge-tree lsm lsm-tree lsmt mit-license rust rust-lang storage-engine

Last synced: about 2 months ago
JSON representation

LSM-based embeddable key-value storage engine written in safe Rust

• Host: GitHub
• URL: https://github.com/fjall-rs/fjall
• Owner: fjall-rs
• License: apache-2.0
• Created: 2023-11-16T13:41:50.000Z (10 months ago)
• Default Branch: main
• Last Pushed: 2024-05-21T17:38:44.000Z (4 months ago)
• Last Synced: 2024-05-22T17:10:26.955Z (4 months ago)
• Topics: database, embeddable, embeddable-database, embedded-database, embedded-kv, key-value-database, key-value-store, kv, kv-database, kv-storage, kv-store, log-structured, log-structured-merge-tree, lsm, lsm-tree, lsmt, mit-license, rust, rust-lang, storage-engine
• Language: Rust
• Homepage:
• Size: 2.35 MB
• Stars: 204
• Watchers: 3
• Forks: 7
• Open Issues: 8
• Metadata Files:
  • Readme: README.md
  • Contributing: CONTRIBUTING.md
  • Funding: .github/FUNDING.yml
  • License: LICENSE-APACHE

Awesome Lists containing this project

README

        


  



.

## Durability

To support different kinds of workloads, Fjall is agnostic about the type of durability

your application needs. After writing data (`insert`, `remove` or committing a write batch), you can choose to call [`Keyspace::persist`](https://docs.rs/fjall/latest/fjall/struct.Keyspace.html#method.persist) which takes a [`PersistMode`](https://docs.rs/fjall/latest/fjall/enum.PersistMode.html) parameter. By default every 1000ms data is fsynced *asynchronously*. Also when dropped, the keyspace will try to persist the journal synchronously.

## Multithreading, Async and Multiprocess

Fjall is internally synchronized for multi-threaded access, so you can clone around the `Keyspace` and `Partition`s as needed, without needing to lock yourself. Common operations like inserting and reading are generally lock free.

For an async example, see the [`tokio`](https://github.com/fjall-rs/fjall/tree/main/examples/tokio) example.

A single keyspace may **not** be loaded in parallel from separate *processes* however.

## Feature flags

#### bloom

Uses bloom filters to reduce disk I/O for non-existing keys. Improves point read performance, but increases memory usage.

*Disabled by default.*

#### single_writer_tx

Allows opening a transactional Keyspace for single-writer (serialized) transactions, allowing RYOW (read-your-own-write), fetch-and-update and other atomic operations.

*Enabled by default.*

## Stable disk format

The disk format is stable as of 1.0.0. Future breaking changes will result in a major version bump and a migration path.

## Examples

[See here](https://github.com/fjall-rs/fjall/tree/main/examples) for practical examples.

And checkout [`Smoltable`](https://github.com/marvin-j97/smoltable), a standalone Bigtable-inspired mini wide-column database using `fjall` as its storage engine.

## Contributing

How can you help?

- [Ask a question](https://github.com/fjall-rs/fjall/discussions/new?category=q-a)

  - or join the Discord server: https://discord.com/invite/HvYGp4NFFk

- [Post benchmarks and things you created](https://github.com/fjall-rs/fjall/discussions/new?category=show-and-tell)

- [Open an issue](https://github.com/fjall-rs/fjall/issues/new) (bug report, weirdness)

- [Open a PR](https://github.com/fjall-rs/fjall/compare)

## License

All source code is licensed under MIT OR Apache-2.0.

All contributions are to be licensed as MIT OR Apache-2.0.