{"id":21182807,"url":"https://github.com/seriousbug/cuttlestore","last_synced_at":"2026-03-06T08:34:14.510Z","repository":{"id":65350825,"uuid":"587604867","full_name":"SeriousBug/cuttlestore","owner":"SeriousBug","description":"A generic rust API for interacting with key-value stores that can be selected at runtime.","archived":false,"fork":false,"pushed_at":"2023-02-25T07:10:13.000Z","size":1696,"stargazers_count":4,"open_issues_count":8,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-08-10T23:12:41.652Z","etag":null,"topics":["keyvalue","rust-crate"],"latest_commit_sha":null,"homepage":"https://seriousbug.github.io/cuttlestore","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SeriousBug.png","metadata":{"files":{"readme":"Readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2023-01-11T06:23:38.000Z","updated_at":"2023-04-14T14:12:09.000Z","dependencies_parsed_at":"2023-02-15T05:16:48.973Z","dependency_job_id":null,"html_url":"https://github.com/SeriousBug/cuttlestore","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SeriousBug%2Fcuttlestore","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SeriousBug%2Fcuttlestore/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SeriousBug%2Fcuttlestore/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SeriousBug%2Fcuttlestore/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SeriousBug","download_url":"https://codeload.github.com/SeriousBug/cuttlestore/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225606620,"owners_count":17495551,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["keyvalue","rust-crate"],"created_at":"2024-11-20T17:58:11.279Z","updated_at":"2026-03-06T08:34:14.452Z","avatar_url":"https://github.com/SeriousBug.png","language":"Rust","readme":"# [Cuttlestore](https://github.com/SeriousBug/Cuttlestore)\n\n[![docs](https://img.shields.io/docsrs/cuttlestore)](https://docs.rs/cuttlestore/latest/cuttlestore/)\n[![tests](https://img.shields.io/github/actions/workflow/status/SeriousBug/cuttlestore/test.yml?label=tests\u0026branch=main)](https://github.com/SeriousBug/cuttlestore/actions/workflows/test.yml)\n[![Test coverage report](https://img.shields.io/codecov/c/github/SeriousBug/cuttlestore)](https://codecov.io/gh/SeriousBug/cuttlestore)\n[![lint checks](https://img.shields.io/github/actions/workflow/status/SeriousBug/cuttlestore/lint.yml?label=lints\u0026branch=main)](https://github.com/SeriousBug/cuttlestore/actions/workflows/lint.yml)\n[![Releases](https://img.shields.io/github/v/release/SeriousBug/cuttlestore?include_prereleases)](https://github.com/SeriousBug/cuttlestore/releases)\n[![MIT license](https://img.shields.io/github/license/SeriousBug/cuttlestore)](https://github.com/SeriousBug/cuttlestore/blob/main/LICENSE.txt)\n\nCuttlestore is a generic API for key-value stores. It allows you to support\nmultiple key-value stores with zero additional effort, and makes it possible to\nswitch between different stores at runtime.\n\n## Example\n\n```rust\nuse cuttlestore::{Cuttlestore, PutOptions};\nuse serde::{Deserialize, Serialize};\nuse std::time::Duration;\n\n#[derive(Debug, Serialize, Deserialize)]\nstruct SelfDestructingMessage {\n    message: String,\n}\n\n#[tokio::main]\nasync fn main() {\n    let store = Cuttlestore::new(\"filesystem://./example-store\")\n                                // or redis, sqlite, in-memory\n        .await\n        .unwrap();\n\n    let mission = SelfDestructingMessage {\n        message: \"Your mission, should you choose to accept it, ...\".to_string(),\n    };\n    store\n        .put_with(\"impossible\", \u0026mission, PutOptions::ttl_secs(60))\n        .await\n        .unwrap();\n\n    // Later\n\n    let value: Option\u003cSelfDestructingMessage\u003e = store.get(\"impossible\").await.unwrap();\n    println!(\"Message says: {value:?}\");\n}\n```\n\n## Supported Backends\n\nCuttlestore currently has support for:\n\n| Name       | Feature            | Connection string | Description                                                                                     | Enabled by default |\n| ---------- | ------------------ | ----------------- | ----------------------------------------------------------------------------------------------- | ------------------ |\n| Redis      | backend-redis      | redis://127.0.0.1 | Backed by Redis. This will get you the best scalability.                                        | Yes                |\n| Sqlite     | backend-sqlite     | sqlite://path     | An sqlite database used as a key-value store. Best performance if scalability is not a concern. | Yes                |\n| Filesystem | backend-filesystem | filesystem://path | Uses files in a folder as a key-value store. Performance depends on your filesystem.            | No                 |\n| In-Memory  | backend-in-memory  | in-memory         | Not persistent, but very high performance. Useful if the store is ephemeral, like a cache.      | Yes                |\n\n## Installing\n\nAdd Cuttlestore to your `Cargo.toml`:\n\n```toml\ncuttlestore = \"0.2\"\n```\n\nIf you want to disable or enable some of the backends, disable the default\nfeatures and pick the ones you want:\n\n```toml\ncuttlestore = { version = \"0.2\", default-features = false, features = [\n  # Only leave redis and sqlite enabled\n  \"backend-redis\",\n  \"backend-sqlite\",\n  # Remember to enable this or `logging-log`\n  \"logging-tracing\",\n] }\n```\n\nYou're now ready to use Cuttlestore! See the example above, check the\n[documentation](https://docs.rs/cuttlestore/latest/cuttlestore/), and find\n[more examples in the repository](https://github.com/SeriousBug/cuttlestore/tree/main/examples).\n\n## Overview\n\n- **Pros:** Cuttlestore is useful if\n  - You want to allow end-users to pick which store to use without recompiling\n  - You are looking for a simple API for a basic key-value store\n- **Cons:** Avoid Cuttlestore if:\n  - You need access to key-value store specific features\n  - You only want to one key-value store, and don't care about switching\n\nFor example, if you are making a self-hostable web application, and you want to\nallow users to pick between using Redis and sqlite depending on their needs, you\ncould use Cuttlestore. Cuttlestore supports both of these backends, and your\nusers could input the connection string in your application settings to pick one\nof these backends. Users with large deployments could pick Redis, and\nsmall-scale users could pick sqlite so they don't have to deal with also\ndeploying Redis.\n\n## Logging\n\nThe library can log errors with both\n[tracing](https://docs.rs/tracing/latest/tracing/) and\n[log](https://docs.rs/log/latest/log/). `tracing` is enabled by default, but you\ncan switch to `log` by enabling the feature:\n\n```toml\ncuttlestore = { version = \"0.2\", default-features = false, features = [\n    \"logging-log\",\n    # remember to enable the backends!\n    \"backend-redis\",\n    \"backend-sqlite\",\n    \"backend-filesystem\",\n    \"backend-in-memory\",\n] }\n```\n\n## Details of backends\n\n### Redis\n\nRedis is generally the best option if you don't mind setting it up. It offers\ngood performance and scalability as you can connect many app servers into the\nsame Redis instance.\n\nCuttlestore has support for TLS, which you can activate by adding an `s` to the\nconnection string like `rediss://127.0.0.1`. You can also change the port you are\nusing by adding `:port` to the end, for example `redis://127.0.0.1:5678`.\n\nCuttlestore has support for ACLs as well. You can enable them by adding them to\nthe connection string. For example, if your username is `agent` and password is\n`47`, you can use the connection string\n`redis://127.0.0.1?username=agent\u0026password=47`.\n\n### Sqlite\n\nCuttlestore can use an sqlite database as a key-value store when using this\nbackend. The database and any tables are automatically created.\n\nThe sqlite database is configured to use write ahead logging, which means it may\ncreate some additional files next to the database file you configure in the\nconnection string. The configuration is also set in a way that there is a small\nchance of losing the last few `put` or `delete` operations if a crash occurs,\nwhich is unfortunately required to bring the performance to a reasonable level.\n\nSqlite doesn't have built-in ttl support, so ttl is supported by periodically\nscanning the database and deleting expired entries on a best-effort basis. This\nscan uses a Tokio task, meaning it will run within your existing Tokio thread\npool.\n\nFor sqlite, you can enable the feature `backend-sqlite-native-tls` or\n`backend-sqlite-rustls` to pick between native TLS or Rustls. `backend-sqlite` is equal to `backend-sqlite-native-tls`.\n\n### Filesystem\n\nCuttlestore can be configured to use a folder as a key value store. When using\nthis backend, the file names in the folder are the keys, and the values are\nstored using a binary encoding within the files.\n\nThe performance largely depends on your filesystem. Durability is similar to\nsqlite: there is a small risk of losing the latest few operations, but data\ncorruption is not expected.\n\nThe ttl feature is supported by periodically scanning the database and deleting\nexpired entries on a best-effort basis. This scan uses a Tokio task, meaning it\nwill run within your existing Tokio thread pool.\n\n#### In-Memory\n\nThe in-memory backend is a multithreaded in-memory key-value store backed by\n[dashmap](https://docs.rs/dashmap/latest/dashmap/index.html).\n\nThe performance is the best, but everything is kept in-memory so there is no\ndurability.\n\n## TTL\n\nThe TTL (time to live) feature allows you to designate values that should only\nexist in the store for a limited amount of time. The values that run out of TTL\nwill be expired and deleted from the store to save space.\n\n```rust\nstore.put_with(\"impossible\", \u0026mission, PutOptions::ttl_secs(60))\n// or\nstore.put_with(\"impossible\", \u0026mission, PutOptions::ttl(Duration::from_secs(60)))\n```\n\nSome backends have built-in support for TTLs (redis). For other backends, the\nTTL support is emulated by periodically running a Tokio task which scans the\nstore and cleans up expired values. This task runs within your existing Tokio\nthread pool. You can configure how often this cleanup task runs using\n`CuttlestoreBuilder`, see the [builder example](https://github.com/SeriousBug/cuttlestore/blob/main/examples/using-builder.rs#L13-L18).\n\nGet and scan operations are guaranteed to never return expired values, but\nexpired values are not necessarily deleted immediately.\n\n## Benchmarks\n\nThere are some [benchmarks to compare the performance of the different backends](https://seriousbug.github.io/cuttlestore/reports/).\nAll the existing benchmarks use small keyspaces so the performance is not\nnecessarily realistic.\n\nThe concurrent benchmarks show you the overall throughput of the backend, while\nthe sequential benchmarks show you the average latency you can expect from each\nrequest.\n\nIn short, these benchmarks show a few things:\n\n- Redis has relatively high latency, around 67 microseconds per operation. In\n  comparison, the second slowest is sqlite with around 13 to 23 microseconds per\n  operation.\n- The filesystem backend offers the best performance both for throughput and\n  latency, but there is a very large spread between the lows and highs. At worst\n  case, it is slower than all other backends.\n- In-memory offers incredible performance, but obviously not durable.\n\nThese benchmarks validate the suggestions listed earlier in the readme. Redis is\na good option if you need scalability, and sqlite is good if scalability is not\na concern. Filesystem can be an option if performance is not critical, but there\nis risk that it will not perform well for large key spaces.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fseriousbug%2Fcuttlestore","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fseriousbug%2Fcuttlestore","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fseriousbug%2Fcuttlestore/lists"}