{"id":22912010,"url":"https://github.com/opencoff/rsbep","last_synced_at":"2025-05-09T01:33:38.242Z","repository":{"id":75757113,"uuid":"56461261","full_name":"opencoff/rsbep","owner":"opencoff","description":"Add Reed Solomon Error correction to archives","archived":false,"fork":false,"pushed_at":"2016-04-17T22:40:44.000Z","size":157,"stargazers_count":22,"open_issues_count":1,"forks_count":3,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-31T20:39:11.280Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://www.thanassis.space/rsbep.html","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/opencoff.png","metadata":{"files":{"readme":"README.md","changelog":"ChangeLog","contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-04-17T22:11:04.000Z","updated_at":"2024-10-29T11:31:56.000Z","dependencies_parsed_at":"2023-06-07T14:45:46.981Z","dependency_job_id":null,"html_url":"https://github.com/opencoff/rsbep","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencoff%2Frsbep","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencoff%2Frsbep/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencoff%2Frsbep/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencoff%2Frsbep/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/opencoff","download_url":"https://codeload.github.com/opencoff/rsbep/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253174320,"owners_count":21865848,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-14T04:19:34.118Z","updated_at":"2025-05-09T01:33:38.234Z","avatar_url":"https://github.com/opencoff.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"WHAT IS THIS?\n=============\n\nThis is a slightly modified version of rsbep0.0.5 which I have\nbeen using to create error-resilient backups of data (able to\nsurvive hard disks' bad sectors, scratched CD/DVD backups, etc)\n\nINSTALL\n=======\n\n\tmake \n\tmake install\n\n...will copy binaries and scripts into your /usr/bin/ \ndirectory. If you wish to change the prefix \nfrom /usr to \"/path/to/whatever\", use...\n\n\tmake\n\tmake install prefix=/path/to/whatever\n\n...and remember to add /path/to/whatever/bin to your PATH.\nRead INSTALL for more details, if you wish.\n\nDATA RECOVERY\n=============\n\nHere's an example of data recovery via the \"freeze.sh\" and \n\"melt.sh\" scripts:\n\n    home:/var/tmp/recovery$ ls -la\n    total 4108\n    drwxr-xr-x 2 ttsiod ttsiod    4096 2008-07-30 22:21 .\n    drwxrwxrwt 5 root   root      4096 2008-07-30 22:21 ..\n    -rw-r--r-- 1 ttsiod ttsiod 4194304 2008-07-30 22:21 data\n\n    home:/var/tmp/recovery$ freeze.sh data \u003e data.shielded\n    home:/var/tmp/recovery$ ls -la\n    total 9204\n    drwxr-xr-x 2 ttsiod ttsiod    4096 2008-07-30 22:21 .\n    drwxrwxrwt 5 root   root      4096 2008-07-30 22:21 ..\n    -rw-r--r-- 1 ttsiod ttsiod 4194304 2008-07-30 22:21 data\n    -rw-r--r-- 1 ttsiod ttsiod 5202000 2008-07-30 22:21 data.shielded\n\n    home:/var/tmp/recovery$ melt.sh data.shielded \u003e data2\n    home:/var/tmp/recovery$ md5sum data data2\n    9440c7d2ff545de1ff340e7a81a53efb  data\n    9440c7d2ff545de1ff340e7a81a53efb  data2\n\n    home:/var/tmp/recovery$ echo Will now create artificial corruption \n    home:/var/tmp/recovery$ echo of 127 times 512 which is 65024 bytes\n\n    home:/var/tmp/recovery$ dd if=/dev/zero of=data.shielded bs=512 count=127 conv=notrunc\n    127+0 records in\n    127+0 records out\n    65024 bytes (65 kB) copied, 0,00026734 seconds, 243 MB/s\n\n    home:/var/tmp/recovery$ melt.sh data.shielded \u003e data3\n\n    rsbep: number of corrected failures   : 64764\n    rsbep: number of uncorrectable blocks : 0\n\n    home:/var/tmp/recovery$ md5sum data data2 data3\n    9440c7d2ff545de1ff340e7a81a53efb  data\n    9440c7d2ff545de1ff340e7a81a53efb  data2\n    9440c7d2ff545de1ff340e7a81a53efb  data3\n\n\nFUSE-based filesystem using this\n================================\n\nThese tools allowed an easy implementation of a Reed-Solomon \nprotected filesystem, using Python FUSE bindings:\n\n    bash$ poorZFS.py -f /reed-solomoned-data /strong\n\nThis command will mount a FUSE-based filesystem in \"/strong\" (using \nthe \"/reed-solomoned-data\" directory to store the actual files and \ntheir \"shielded\" versions). Any file you create in /strong, will \nin fact exist under \"/reed-solomoned-data\", and will also be shielded \nthere (via \"freeze.sh\"). When opening for reading any file in /strong, \ndata corruption is detected (via \"melt.sh\") and in case of corruption \nthe file will be corrected using the Reed-Solomon \"shielded\" version \nof the file (which is stored alongside the original, and named as\noriginalFilename.frozen.RS).\n\nI coded this using Python-FUSE in a couple of hours on a boring Sunday \nafternoon, so don't trust your bank account data with it... It's just\na proof of concept (not to mention dog-slow). Still, if your machine\nis only equipped with one drive, this will in fact transparently shield\nyou against any sudden appearance of your drive's bad sectors.\n\nAnd just note that I coded this filesystem adding 20 or so lines of\nPython (spawning my freeze/melt scripts) into the Python FUSE basic \nexample. Anyone who has ever coded a filesystem driver for Windows knows \nwhy this justifies a heart attack - FUSE (and Python FUSE) rock!\n\nCHANGES from original rsbep\n===========================\n\n- The original version wrote 3 parameters of the Reed-Solomon algorithm\n  as a single line before the \"shielded\" data, and this made the stream \n  fragile (if this information was lost, decoding failed...) \n\n- It uses a default value of 16*255=4080 for parameter R, \n  and it can thus tolerate 4080*16=65280 consecutive bytes \n  to be lost anywhere in the stream, and still recover...\n\n- It adds file size information in the shielded stream, so the recovery \n  process re-creates an exact copy of the original.\n\n- I added autoconf/automake support, to detect whether a fast 32bit x86 \n  asm version can be used and otherwise fall back to a plain C (slow) \n  implementation. The tools thus compile and install cleanly on many \n  operating systems (Linux, Mac OS/X, Free/Net/OpenBSD, even Windows\n  with Cygwin).\n\n- Python-FUSE support.\n\nFEEDBACK\n========\nNo thanks :-)\nIt works as is for me, if you don't like it, fix it yourself.\nIt's useful to me, hope it proves useful to you too...\n\nThanassis Tsiodras, Dr.-Ing\nttsiodras at removeThis gmail dot com (I hate spam)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopencoff%2Frsbep","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopencoff%2Frsbep","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopencoff%2Frsbep/lists"}