{"id":13875999,"url":"https://github.com/sourcefrog/conserve","last_synced_at":"2025-05-15T23:06:08.894Z","repository":{"id":47065245,"uuid":"6107385","full_name":"sourcefrog/conserve","owner":"sourcefrog","description":"🌲 Robust file backup tool in Rust","archived":false,"fork":false,"pushed_at":"2025-05-15T13:58:15.000Z","size":3469,"stargazers_count":275,"open_issues_count":50,"forks_count":21,"subscribers_count":11,"default_branch":"main","last_synced_at":"2025-05-15T15:21:51.314Z","etag":null,"topics":["backup","linux","macos","restore","robust","rust","windows"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sourcefrog.png","metadata":{"files":{"readme":"README.md","changelog":"NEWS.md","contributing":"CONTRIBUTING.md","funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2012-10-06T22:30:55.000Z","updated_at":"2025-05-15T13:57:31.000Z","dependencies_parsed_at":"2023-11-23T22:32:56.628Z","dependency_job_id":"b855b4bb-f4c5-4626-a0e3-dae5a7e43004","html_url":"https://github.com/sourcefrog/conserve","commit_stats":{"total_commits":2404,"total_committers":12,"mean_commits":"200.33333333333334","dds":"0.021214642262895156","last_synced_commit":"ed09bf723a84e8afe9cb7685dc5b148ce3b7fd0f"},"previous_names":[],"tags_count":33,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sourcefrog%2Fconserve","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sourcefrog%2Fconserve/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sourcefrog%2Fconserve/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sourcefrog%2Fconserve/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sourcefrog","download_url":"https://codeload.github.com/sourcefrog/conserve/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254364613,"owners_count":22058955,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backup","linux","macos","restore","robust","rust","windows"],"created_at":"2024-08-06T06:00:55.613Z","updated_at":"2025-05-15T23:06:03.853Z","avatar_url":"https://github.com/sourcefrog.png","language":"Rust","readme":"# Conserve: a robust backup program\n\n\u003chttps://github.com/sourcefrog/conserve/\u003e\n\n[![GitHub build status](https://github.com/sourcefrog/conserve/workflows/Rust/badge.svg?branch=main)](https://github.com/sourcefrog/conserve/actions?query=workflow%3ARust)\n[![crates.io](https://img.shields.io/crates/v/conserve.svg)](https://crates.io/crates/conserve)\n![Maturity: Beta](https://img.shields.io/badge/maturity-beta-yellow.svg)\n\n\u003c!-- [![Join the chat at https://gitter.im/sourcefrog/conserve](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/sourcefrog/conserve?utm_source=badge\u0026utm_medium=badge\u0026utm_campaign=pr-badge\u0026utm_content=badge) --\u003e\n\nConserve's [guiding principles](doc/manifesto.md):\n\n- **Safe**: Conserve is written in [Rust][rust], a fast systems programming\n  language with compile-time guarantees about types, memory safety, and\n  concurrency. Conserve uses a\n  [conservative log-structured format](doc/format.md).\n\n- **Robust**: If one file is corrupted in storage or due to a bug in Conserve,\n  or if the backup is interrupted, you can still restore what was written.\n  (Conserve doesn't need a large transaction to complete for data to be\n  accessible.)\n\n- **Careful**: Backup data files are never touched or altered after they're\n  written, unless you choose to purge them.\n\n- **When you need help now**: Restoring a subset of a large backup is fast,\n  because it doesn't require reading the whole backup.\n\n- **Always making progress**: Even if the backup process or its network\n  connection is repeatedly killed, Conserve can quickly pick up where it left\n  off and make forward progress.\n\n- **Ready for today**: The storage format is fast and reliable on on\n  high-latency, limited-capability, unlimited-capacity, eventually-consistent\n  cloud object storage.\n\n- **Fast**: Conserve exploits Rust's _fearless concurrency_ to make full use of\n  multiple cores and IO bandwidth. (In the current release there's still room to\n  add more concurrency.)\n\n- **Portable**: Conserve is tested on Windows, Linux (x86 and ARM), and OS X.\n\n## Quick start guide\n\nConserve storage is within an _archive_ directory created by `conserve init`:\n\n    conserve init /backup/home.cons\n\n`conserve backup` copies a source directory into a new _version_ within the\narchive. Conserve copies files, directories, and (on Unix) symlinks. If the\n`conserve backup` command completes successfully (copying the whole source\ntree), the backup is considered _complete_.\n\n    conserve backup /backup/home.cons ~ --exclude /.cache\n\n`conserve diff` shows what's different between an archive and a source\ndirectory. It should typically be given the same `--exclude` options as were\nused to make the backup.\n\n    conserve diff /backup/home.cons ~ --exclude /.cache\n\n`conserve versions` lists the versions in an archive, whether or not the backup\nis _complete_, the time at which the backup started, and the time taken to\ncomplete it. Each version is identified by a name starting with `b`.\n\n    $ conserve versions /backup/home.cons\n    b0000                      complete   2016-11-19T07:30:09+11:00     71s\n    b0001                      incomplete 2016-11-20T06:26:46+11:00\n    b0002                      incomplete 2016-11-20T06:30:45+11:00\n    b0003                      complete   2016-11-20T06:42:13+11:00    286s\n    b0004                      complete   2016-12-01T07:08:48+11:00     84s\n    b0005                      complete   2016-12-18T02:43:59+11:00      4s\n\n`conserve ls` shows all the files in a particular version. Like all commands\nthat read a band from an archive, it operates on the most recent by default, and\nyou can specify a different version using `-b`. (You can also omit leading zeros\nfrom the backup version.)\n\n    conserve ls -b b0 /backup/home.cons | less\n\n`conserve restore` copies a version back out of an archive:\n\n    conserve restore /backup/home.cons /tmp/trial-restore\n\n`conserve validate` checks the integrity of an archive:\n\n    conserve validate /backup/home.cons\n\n`conserve delete` deletes specific named backups from an archive:\n\n    conserve delete /backup/home.cons -b b1\n\n## Exclusions\n\nThe `--exclude GLOB` option can be given to commands that operate on files,\nincluding `backup`, `restore`, `ls` and `list-source`.\n\nA `/` at the start of the exclusion pattern anchors it to the top of the backup\ntree (not the root of the filesystem.) `**` recursively matches any number of\ndirectories. `*.o` matches anywhere in the tree.\n\n`--exclude-from` reads exclusion patterns from a file, one per line, ignoring\nleading and trailing whitespace, and skipping comment lines that start with a\n`#`.\n\nThe syntax is comes from the Rust [globset](https://docs.rs/globset/#syntax)\ncrate.\n\nDirectories marked with [`CACHEDIR.TAG`](https://bford.info/cachedir/) are\nautomatically excluded from backups.\n\n## S3 support\n\nFrom 23.9 Conserve supports storing backups in Amazon S3. AWS IAM credentials are\nread from the standard sources: the environment, config file, or, on EC2, the instance metadata service.\n\nTo use this, just specify an S3 URL for the archive location. The bucket must already exist.\n\n    conserve init s3://my-bucket/\n    conserve backup s3://my-bucket/ ~\n\nFiles are written in the `INTELLIGENT_TIERING` storage class.\n\n(This should work on API-compatible services but has not been tested; experience reports are welcome.)\n\n## Install\n\nTo build Conserve you need [Rust][rust] and a C compiler that can be used by\nRust.\n\nTo install the most recent release from crates.io, run\n\n    cargo install conserve\n\nTo install from a git checkout, run\n\n    cargo install -f --path .\n\n[rust]: https://rustup.rs/\n\n### Optional features\n\nThe following features are enabled by default, but can be turned off with `cargo install --no-default-features` if they are not needed:\n\n- `s3`: support for storing backups in Amazon S3 (or compatible services)\n- `sftp`: support for storing backups on SFTP servers, addressed with `sftp://` URLs\n\n### Arch Linux\n\nTo install from from available\n[AUR packages](https://aur.archlinux.org/packages/?O=0\u0026SeB=nd\u0026K=Robust+portable+backup+tool+written\u0026outdated=\u0026SB=n\u0026SO=a\u0026PP=50\u0026do_Search=Go),\nuse an [AUR helper](https://wiki.archlinux.org/index.php/AUR_helpers):\n\n    yay -S conserve\n\n## More documentation\n\n- [A comparison to other backup systems][comparison]\n\n[comparison]: https://github.com/sourcefrog/conserve/wiki/Compared-to-others\n\n- [Software and format versioning](doc/versioning.md)\n\n- [Archive format](doc/format.md)\n\n- [Design](doc/design.md)\n\n- [Release notes](NEWS.md)\n\n## Performance on Windows\n\nWindows Defender and Windows Search Indexing can severely slow down any program that does intensive file IO, including Conserve. I recommend you exclude the backup directory from both systems.\n\n## Project status\n\nConserve is at a reasonable level of maturity; the format is stable and the basic features are complete. I have used it as a primary backup system for over a year.\nThere is still room for several performance improvements and features.\n\nThe current data format (called \"0.6\") will be readable by future releases for at least two years.\n\nBe aware Conserve is developed as a part-time non-commercial project and there's no guarantee of support or reliability. Bug reports are welcome but I cannot promise they will receive a resolution within any particular time frame.\n\n## Licence and non-warranty\n\nCopyright 2012-2023 Martin Pool.\n\nThis program is free software; you can redistribute it and/or modify it under\nthe terms of the GNU General Public License as published by the Free Software\nFoundation; either version 2 of the License, or (at your option) any later\nversion.\n\nThis program is distributed in the hope that it will be useful, but WITHOUT ANY\nWARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A\nPARTICULAR PURPOSE. See the GNU General Public License for more details.\n","funding_links":[],"categories":["windows","Rust","macos","Backup Software"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsourcefrog%2Fconserve","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsourcefrog%2Fconserve","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsourcefrog%2Fconserve/lists"}