{"id":14155163,"url":"https://github.com/slatedb/slatedb","last_synced_at":"2025-05-13T18:07:12.364Z","repository":{"id":252995105,"uuid":"777912773","full_name":"slatedb/slatedb","owner":"slatedb","description":"A cloud native embedded storage engine built on object storage.","archived":false,"fork":false,"pushed_at":"2025-05-09T17:33:50.000Z","size":1529,"stargazers_count":1972,"open_issues_count":109,"forks_count":105,"subscribers_count":18,"default_branch":"main","last_synced_at":"2025-05-09T17:45:41.570Z","etag":null,"topics":["database","embedded-database","lsm-tree","object-storage","rocksdb","rust","storage-engine"],"latest_commit_sha":null,"homepage":"https://slatedb.io","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/slatedb.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":"GOVERNANCE.md","roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"open_collective":"commonhaus-foundation"}},"created_at":"2024-03-26T18:28:31.000Z","updated_at":"2025-05-09T16:42:09.000Z","dependencies_parsed_at":"2024-12-09T23:29:35.301Z","dependency_job_id":"edff58d5-5e2a-4f9c-aacd-f7dcd749667b","html_url":"https://github.com/slatedb/slatedb","commit_stats":{"total_commits":236,"total_committers":30,"mean_commits":7.866666666666666,"dds":0.6228813559322034,"last_synced_commit":"ce2ddee5eaa28015434875157c182a959374c04b"},"previous_names":["slatedb/slatedb"],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slatedb%2Fslatedb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slatedb%2Fslatedb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slatedb%2Fslatedb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slatedb%2Fslatedb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/slatedb","download_url":"https://codeload.github.com/slatedb/slatedb/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254000848,"owners_count":21997441,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","embedded-database","lsm-tree","object-storage","rocksdb","rust","storage-engine"],"created_at":"2024-08-17T08:02:19.088Z","updated_at":"2025-05-13T18:07:12.352Z","avatar_url":"https://github.com/slatedb.png","language":"Rust","readme":"\u003ca href=\"https://slatedb.io\"\u003e\n  \u003cimg src=\"https://github.com/slatedb/slatedb-website/blob/main/assets/png/gh-banner.png?raw=true\" alt=\"SlateDB\" width=\"100%\"\u003e\n\u003c/a\u003e\n\n\u003ca href=\"https://crates.io/crates/slatedb\"\u003e![Crates.io Version](https://img.shields.io/crates/v/slatedb?style=flat-square)\u003c/a\u003e\n![GitHub License](https://img.shields.io/github/license/slatedb/slatedb?style=flat-square)\n\u003ca href=\"https://slatedb.io\"\u003e![slatedb.io](https://img.shields.io/badge/site-slatedb.io-00A1FF?style=flat-square)\u003c/a\u003e\n\u003ca href=\"https://discord.gg/mHYmGy5MgA\"\u003e![Discord](https://img.shields.io/discord/1232385660460204122?style=flat-square)\u003c/a\u003e\n\u003ca href=\"https://docs.rs/slatedb/latest/slatedb/\"\u003e![Docs](https://img.shields.io/badge/docs-docs.rs-00A1FF?style=flat-square)\u003c/a\u003e\n\n## Introduction\n\n[SlateDB](https://slatedb.io) is an embedded storage engine built as a [log-structured merge-tree](https://en.wikipedia.org/wiki/Log-structured_merge-tree). Unlike traditional LSM-tree storage engines, SlateDB writes data to object storage (S3, GCS, ABS, MinIO, Tigris, and so on). Leveraging object storage allows SlateDB to provide bottomless storage capacity, high durability, and easy replication. The trade-off is that object storage has a higher latency and higher API cost than local disk.\n\nTo mitigate high write API costs (PUTs), SlateDB batches writes. Rather than writing every `put()` call to object storage, MemTables are flushed periodically to object storage as a string-sorted table (SST). The flush interval is configurable.\n\n`put()` returns a `Future` that resolves when the data is durably persisted. Clients that prefer lower latency at the cost of durability can instead use `put_with_options` with `await_durable` set to `false`.\n\nTo mitigate read latency and read API costs (GETs), SlateDB will use standard LSM-tree caching techniques: in-memory block caches, compression, bloom filters, and local SST disk caches.\n\nCheckout [slatedb.io](https://slatedb.io) to learn more.\n\n## Get Started\n\nAdd the following to your `Cargo.toml`:\n\n```toml\n[dependencies]\nslatedb = \"*\"\ntokio = \"*\"\n```\n\nThen you can use SlateDB in your Rust code:\n\n```rust\nuse slatedb::{Db, SlateDBError};\nuse slatedb::object_store::{ObjectStore, memory::InMemory};\nuse std::sync::Arc;\n\n#[tokio::main]\nasync fn main() -\u003e Result\u003c(), SlateDBError\u003e {\n    // Setup\n    let object_store: Arc\u003cdyn ObjectStore\u003e = Arc::new(InMemory::new());\n    let kv_store = Db::open(\"/tmp/test_kv_store\", object_store).await?;\n\n    // Put\n    let key = b\"test_key\";\n    let value = b\"test_value\";\n    kv_store.put(key, value).await?;\n\n    // Get\n    assert_eq!(\n        kv_store.get(key).await?,\n        Some(\"test_value\".into())\n    );\n\n    // Delete\n    kv_store.delete(key).await?;\n    assert!(kv_store.get(key).await?.is_none());\n\n    kv_store.put(b\"test_key1\", b\"test_value1\").await?;\n    kv_store.put(b\"test_key2\", b\"test_value2\").await?;\n    kv_store.put(b\"test_key3\", b\"test_value3\").await?;\n    kv_store.put(b\"test_key4\", b\"test_value4\").await?;\n\n    // Scan over unbound range\n    let mut iter = kv_store.scan::\u003cVec\u003cu8\u003e, _\u003e(..).await?;\n    let mut count = 1;\n    while let Ok(Some(item)) = iter.next().await {\n        assert_eq!(\n            item.key,\n            format!(\"test_key{count}\").into_bytes()\n        );\n        assert_eq!(\n            item.value,\n            format!(\"test_value{count}\").into_bytes()\n        );\n        count += 1;\n    }\n\n    // Scan over bound range\n    let mut iter = kv_store.scan(\"test_key1\"..=\"test_key2\").await?;\n    assert_eq!(\n        iter.next().await?,\n        Some((b\"test_key1\", b\"test_value1\").into())\n    );\n    assert_eq!(\n        iter.next().await?,\n        Some((b\"test_key2\", b\"test_value2\").into())\n    );\n\n    // Seek ahead to next key\n    let mut iter = kv_store.scan::\u003cVec\u003cu8\u003e, _\u003e(..).await?;\n    let next_key = b\"test_key4\";\n    iter.seek(next_key).await?;\n    assert_eq!(\n        iter.next().await?,\n        Some((b\"test_key4\", b\"test_value4\").into())\n    );\n    assert_eq!(iter.next().await?, None);\n\n    // Close\n    kv_store.close().await?;\n\n    Ok(())\n}\n```\n\nSlateDB uses the [`object_store`](https://docs.rs/object_store/latest/object_store/) crate to interact with object storage, and therefore supports any object storage that implements the `ObjectStore` trait. You can use the crate in your project to interact with any object storage that implements the `ObjectStore` trait. SlateDB also re-exports the [`object_store`](https://docs.rs/object_store/latest/object_store/) crate for your convenience.\n\n## Documentation\n\nVisit [slatedb.io](https://slatedb.io) to learn more.\n\n## Features\n\nSlateDB is currently in the early stages of development. It is not yet ready for production use.\n\n- [x] Basic API (get, put, delete)\n- [x] SSTs on object storage\n- [x] Range queries ([#8](https://github.com/slatedb/slatedb/issues/8))\n- [x] Block cache ([#15](https://github.com/slatedb/slatedb/issues/15))\n- [x] Disk cache ([#9](https://github.com/slatedb/slatedb/issues/9))\n- [x] Compression ([#10](https://github.com/slatedb/slatedb/issues/10))\n- [x] Bloom filters ([#11](https://github.com/slatedb/slatedb/issues/11))\n- [x] Manifest persistence ([#14](https://github.com/slatedb/slatedb/issues/14))\n- [x] Compaction ([#7](https://github.com/slatedb/slatedb/issues/7))\n- [ ] Transactions\n- [ ] Merge operator ([#328](https://github.com/slatedb/slatedb/issues/328))\n- [x] Clones ([#49](https://github.com/slatedb/slatedb/issues/49))\n\n## License\n\nSlateDB is licensed under the Apache License, Version 2.0.\n\n## Foundation\n\nSlateDB is a member of the [Commonhaus Foundation](https://www.commonhaus.org/).\n\n\u003cpicture\u003e\n  \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"https://github.com/commonhaus/artwork/blob/main/foundation/brand/png/CF_logo_horizontal_single_reverse_200px.png?raw=true\"\u003e\n  \u003cimg src=\"https://github.com/commonhaus/artwork/blob/main/foundation/brand/png/CF_logo_horizontal_single_default_200px.png?raw=true\"\u003e\n\u003c/picture\u003e\n\n","funding_links":["https://opencollective.com/commonhaus-foundation"],"categories":["Rust","rust","\u003ca name=\"Rust\"\u003e\u003c/a\u003eRust"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fslatedb%2Fslatedb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fslatedb%2Fslatedb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fslatedb%2Fslatedb/lists"}