{"id":13875956,"url":"https://github.com/lostatc/acid-store","last_synced_at":"2025-07-16T10:32:14.039Z","repository":{"id":49054049,"uuid":"237480481","full_name":"lostatc/acid-store","owner":"lostatc","description":"[UNMAINTAINED] A transactional and deduplicating virtual file system","archived":true,"fork":false,"pushed_at":"2024-03-03T15:21:37.000Z","size":3202,"stargazers_count":95,"open_issues_count":2,"forks_count":10,"subscribers_count":5,"default_branch":"main","last_synced_at":"2024-11-15T09:52:17.909Z","etag":null,"topics":["acid","deduplication","encryption","filesystem","fuse","rclone","redis","rust","s3","sftp","sqlite","storage"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lostatc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-01-31T17:25:00.000Z","updated_at":"2024-11-02T08:24:44.000Z","dependencies_parsed_at":"2024-11-19T02:01:50.840Z","dependency_job_id":null,"html_url":"https://github.com/lostatc/acid-store","commit_stats":{"total_commits":709,"total_committers":3,"mean_commits":"236.33333333333334","dds":0.04090267983074758,"last_synced_commit":"aa7940146b9da7f66c82bfbed64216811d8959a3"},"previous_names":[],"tags_count":22,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lostatc%2Facid-store","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lostatc%2Facid-store/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lostatc%2Facid-store/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lostatc%2Facid-store/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lostatc","download_url":"https://codeload.github.com/lostatc/acid-store/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":226122303,"owners_count":17576920,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["acid","deduplication","encryption","filesystem","fuse","rclone","redis","rust","s3","sftp","sqlite","storage"],"created_at":"2024-08-06T06:00:52.685Z","updated_at":"2024-11-24T03:31:33.522Z","avatar_url":"https://github.com/lostatc.png","language":"Rust","funding_links":[],"categories":["Rust","sqlite"],"sub_categories":[],"readme":"[![Tests Workflow Status (main)](https://img.shields.io/github/actions/workflow/status/lostatc/acid-store/test.yaml?branch=main\u0026label=Tests\u0026style=for-the-badge\u0026logo=github)](https://github.com/lostatc/acid-store/actions/workflows/test.yaml)\n[![Codecov](https://img.shields.io/codecov/c/github/lostatc/acid-store?logo=codecov\u0026style=for-the-badge)](https://app.codecov.io/gh/lostatc/acid-store)\n[![Crates.io](https://img.shields.io/crates/v/acid-store?logo=rust\u0026style=for-the-badge)](https://crates.io/crates/acid-store)\n[![docs.rs](https://img.shields.io/docsrs/acid-store?logo=docsdotrs\u0026style=for-the-badge)](https://docs.rs/acid-store)\n\n⚠ **UNMAINTAINED**\n\n*I am no longer maintaining this project.*\n\n# acid-store\n\nacid-store is a Rust library for secure, deduplicated, and transactional data\nstorage.\n\nThis library provides abstractions for data storage over a number of storage\nbackends. You can turn any storage backend into an encrypted and deduplicated\nobject store, persistent collection, or virtual file system (which can be\nmounted via FUSE).\n\nOut of the box, this library supports the local file system, SQLite, Redis,\nAmazon S3, SFTP, and many cloud providers as storage backends. Storage backends\nare easy to implement, and this library builds on top of them to provide\nencryption, compression, content-based deduplication, locking, and atomic\ntransactions.\n\nFor details and examples, see the [documentation](https://docs.rs/acid-store).\n\n⚠ This project is experimental! ⚠\n\nThis project experiences frequent breaking API changes and hasn't seen\nsignificant real-world usage. This project is not ready for use in production\nenvironments. Also keep in mind that this code has not been audited for\nsecurity.\n\n## Features\n\n- Optional encryption of all data and metadata using XChaCha20-Poly1305 and\n  Argon2, via [libsodium](https://download.libsodium.org/doc/)\n- Optional compression using LZ4\n- Optional content-based deduplication\n- Supports packing data into fixed-size blocks to avoid metadata leakage when\n  using encryption\n- Integrity checking of data and metadata using checksums and (if encryption is\n  enabled) AEAD\n- Locking protects against concurrent access from multiple clients\n- Copy-on-write semantics\n- New storage backends are easy to implement\n\n### Abstractions\n\nThis library provides the following abstractions for data storage.\n\n- An object store which maps keys to seekable binary blobs\n- A virtual file system which can be mounted via FUSE and supports file\n  metadata, special files, sparse files, hard links, and importing and exporting\n  files to the local file system\n- A persistent, heterogeneous, map-like collection\n\n### Backends\n\nThis library provides the following storage backends out of the box.\n\n- Local file system directory\n- SQLite\n- Redis\n- Amazon S3\n- SFTP\n- Cloud storage via [rclone](https://rclone.org/)\n- In-Memory\n\n## Benchmarks\n\nThe following results show read and write speeds using an in-memory storage\nbackend. You can run the benchmarks yourself by running `cargo bench --features\n'encryption'`.\n\n### Specs\n\n| Spec      | Value                |\n| --------- | -------------------- |\n| Processor | Ryzen 5 1600x        |\n| Memory    | 32 GB DDR4 (3200MHz) |\n| OS        | Linux 5.11           |\n\n### Results\n\n| Chunking | Packing | Encryption         | Compression | Read       | Write      |\n| -------- | ------- | ------------------ | ----------- | ---------- | ---------- |\n| Fixed    | None    | None               | None        | 6090 MiB/s | 1920 MiB/s |\n| ZPAQ     | None    | None               | None        | 2670 MiB/s | 520 MiB/s  |\n| Fixed    | Fixed   | XChaCha20-Poly1305 | None        | 870 MiB/s  | 610 MiB/s  |\n| ZPAQ     | Fixed   | XChaCha20-Poly1305 | None        | 840 MiB/s  | 300 MiB/s  |\n\n## MSRV Policy\n\nThe last two stable Rust releases are supported. Older releases may be supported\nas well.\n\nThe MSRV will only be increased when necessary to take advantage of new Rust\nfeatures—not every time there is a new Rust release. An increase in the MSRV\nwill be accompanied by a minor semver bump if \u003e=1.0.0 or a patch semver bump if\n\u003c1.0.0.\n\nThis policy was added with v0.13.0.\n\n## Semver Policy\n\nPrior to version 1.0.0, breaking changes will be accompanied by a minor version\nbump, and new features and bug fixes will be accompanied by a patch version\nbump.\n\nThis policy was added with v0.13.0.\n\n## Similar Projects\n\nHere are some similar projects to check out:\n\n- [Persy](https://persy.rs/)\n- [ZboxFS](https://zbox.io/fs/)\n\n## Copyright\n\nCopyright © 2019-2023 Wren Powell\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n    http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flostatc%2Facid-store","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flostatc%2Facid-store","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flostatc%2Facid-store/lists"}