{"id":13587150,"url":"https://github.com/darrenldl/blockyarchive","last_synced_at":"2025-04-07T19:30:55.687Z","repository":{"id":33790107,"uuid":"111792172","full_name":"darrenldl/blockyarchive","owner":"darrenldl","description":"Blocky archive - multithreaded archiver offering bit rot protection and sector level recoverability","archived":true,"fork":false,"pushed_at":"2022-06-06T19:36:17.000Z","size":8449,"stargazers_count":92,"open_issues_count":14,"forks_count":4,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-05-01T15:10:58.206Z","etag":null,"topics":["archive","backup","bitrot","blkar","blocky-archive","blockyarchive","burst-sector-errors","data-recovery","fec","forward-error-correction","recovery","reed-solomon","reedsolomon","rsbx","rust","sbx","sector-level-recovery","seqbox"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/darrenldl.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-11-23T09:53:01.000Z","updated_at":"2024-03-14T05:06:28.000Z","dependencies_parsed_at":"2022-07-24T23:46:42.161Z","dependency_job_id":null,"html_url":"https://github.com/darrenldl/blockyarchive","commit_stats":null,"previous_names":["darrenldl/rust-seqbox"],"tags_count":42,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/darrenldl%2Fblockyarchive","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/darrenldl%2Fblockyarchive/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/darrenldl%2Fblockyarchive/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/darrenldl%2Fblockyarchive/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/darrenldl","download_url":"https://codeload.github.com/darrenldl/blockyarchive/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223290404,"owners_count":17120908,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["archive","backup","bitrot","blkar","blocky-archive","blockyarchive","burst-sector-errors","data-recovery","fec","forward-error-correction","recovery","reed-solomon","reedsolomon","rsbx","rust","sbx","sector-level-recovery","seqbox"],"created_at":"2024-08-01T15:06:02.982Z","updated_at":"2024-11-06T05:31:16.809Z","avatar_url":"https://github.com/darrenldl.png","language":"Rust","funding_links":["http://ko-fi.com/darrenldl"],"categories":["Rust","rust","Long-term data archiving"],"sub_categories":["Download automation"],"readme":"# blockyarchive\n\n[![Build Status](https://travis-ci.org/darrenldl/blockyarchive.svg?branch=master)](https://travis-ci.org/darrenldl/blockyarchive)\n[![codecov](https://codecov.io/gh/darrenldl/blockyarchive/branch/master/graph/badge.svg)](https://codecov.io/gh/darrenldl/blockyarchive)\n[![Crates](https://img.shields.io/crates/v/blkar.svg)](https://crates.io/crates/blkar)\n[![dependency status](https://deps.rs/repo/github/darrenldl/blockyarchive/status.svg)](https://deps.rs/repo/github/darrenldl/blockyarchive)\n[![Gitter chat](https://badges.gitter.im/blockyarchive/gitter.png)](https://gitter.im/blockyarchive/community)\n\n[Documentation](https://github.com/darrenldl/blockyarchive/wiki)\n\nBlockyarchive/blkar (pronounced \"bloc-kar\") is a multithreaded archiver written in Rust that offers bit rot protection, and makes it easier to recover archived data from failing storage devices.\n\n### Demo\n\n[![asciicast](https://asciinema.org/a/256057.svg)](https://asciinema.org/a/256057)\n\n### How does it work?\n\nblkar encodes your data into SeqBox and EC-SeqBox archives. Both formats facilitate data recovery, but only EC-SeqBox provides data repair capability.\n\n### What are SeqBox and EC-SeqBox?\n\nSeqBox is a single-file archive format designed by [Marco Pontello](https://github.com/MarcoPon) that facilitates sector level data recovery for when file system metadata is corrupted/missing, while the archive itself still exists as a normal file on file system. Please visit the official [SeqBox](https://github.com/MarcoPon/SeqBox) repo for the original implementation and technical details on this.\n\nError-correcting SeqBox (or EC-SeqBox for short) is an extended version of SeqBox developed by [Darren Ldl](https://github.com/darrenldl) for this project, introducing forward error correction via Reed-Solomon erasure code.\n\nBlockyarchive/blkar was formerly known as rust-SeqBox/rsbx prior to renaming.\n\n### Features overall\n\n- Data recovery that does not depend on file system metadata (sector level recovery)\n    - This allows data recovery even when data is fragmented and out of order\n- Supports error correction (via Reed-Solomon erasure code) for EC-SeqBox\n- Supports burst (sector) error resistance for EC-SeqBox\n    - This is done via an interleaving block arrangement scheme. It is mainly to address the data repair limitation of the simple archive design.\n    - More complex archive designs such as PAR2 can repair burst errors without any extra arrangement scheme, but they are also vastly more complex than EC-SeqBox\n- Multithreaded\n    - A lot of operations involved in everyday workflow are written to take advantage of multi-core CPU to provide high performance\n- JSON mode\n    - Outputs information in JSON format instead of human readable text, allowing easy integration with scripts\n\n### Limitations\n\n- Only a single file is supported for encoding as SeqBox and EC-SeqBox are both single-file archive formats\n    - However, blkar may still be usable when you have multiple files, as blkar supports taking input from stdin during encoding, and also supports outputting to stdout during decoding\n    - This means if you have an archiver that supports bundling and unbundling on the fly with pipes, like tar, you can combine the use of the archiver and blkar into one encoding and decoding step\n\n### Getting started\n\n**Installation**\n\nblkar is available via [AUR](https://aur.archlinux.org/packages/blkar), [GitHub releases](https://github.com/darrenldl/blockyarchive/releases) or `cargo`\n\n```\ncargo install blkar\n```\n\n**Usage guides \u0026 screencasts \u0026 other resources**\n\nThe [wiki](https://github.com/darrenldl/blockyarchive/wiki) contains comprehensive guides and resources.\n\n### Goals and status\n\nAs blkar is to be used largely as a backup utility, security/robustness of the code will be prioritised over apparent performance.\n\nThis project has reached its intended feature completeness, so no active development for new features will occur. However, this project is still actively looked after, i.e. I will respond to PRs, issues, and emails, will consider feature requests, respond to bug reports quickly, and so on.\n\nIn other words, this is a completed project with respect to its original scope, but it is not abandoned.\n\n### Links\n\n[Comparison to the original SeqBox implementation/design](COMPARISON.md)\n\n[Changelog](CHANGELOG.md)\n\n[SBX format](SBX_FORMAT.md) (EC-SeqBox is also specified in this document)\n\n[blkar specs](BLKAR_SPECS.md)\n\n### Contributions\n\nContributions are welcome. Note that by submitting contributions, you agree to license your work under the same license used by this project as stated in the LICENSE file.\n\n### Acknowledgement\n\nI would like to thank [Marco](https://github.com/MarcoPon) (the official SeqBox author) for discussing and clarifying aspects of his project, and also providing of test data during development of osbx. I would also like to thank him for his feedback on the numbering of the error correction enabled ECSBX versions (versions 17, 18, 19).\n\nI would like to thank [Ming](https://github.com/mdchia/) for his feedback on the documentation, UX design, and several other general aspects of the osbx project, of which most of the designs are carried over to blkar, and also his further feedback on this project as well.\n\nThe design of the readable rate in progress report text is copied from [Arch Linux pacman](https://wiki.archlinux.org/index.php/Pacman)'s progress bar design.\n\nThe design of block set interleaving arrangement in RS enabled versions is heavily inspired by [Thanassis Tsiodras's design of RockFAT](https://www.thanassis.space/RockFAT.html). The interleaving provides resistance against burst sector errors.\n\n### Donation\n\n**Note**: Donation will **NOT** fuel development of new features. As mentioned above, this project is meant to be stable, well tested and well maintained, but normally I am not actively adding new features to it.\n\nIf blockyarchive has been useful to you, and you would like to donate to me for the development effort, you can donate through [here](http://ko-fi.com/darrenldl).\n\n### License\n\n**Libcrc code**\n\nThe crcccitt code is translated from the C implementation in [libcrc](https://github.com/lammertb/libcrc) and is under the same MIT License as used by libcrc and as stated in libcrc source code. The license text of the crcccitt.c is copied over to `crc-ccitt/build.rs`, `crc-ccitt/src/lib.rs`, `build.rs` and `src/crc_ccitt.rs` as well.\n\n**Official SeqBox code**\n\nThe following files in tests folder copied from official SeqBox are under its license, which is MIT as of time of writing\n\n- tests/SeqBox/*\n\nAll remaining files are distributed under the MIT license as stated in the LICENSE file.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdarrenldl%2Fblockyarchive","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdarrenldl%2Fblockyarchive","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdarrenldl%2Fblockyarchive/lists"}