{"id":19491011,"url":"https://github.com/quark-zju/outagefs","last_synced_at":"2025-08-16T10:49:40.726Z","repository":{"id":57650942,"uuid":"289777585","full_name":"quark-zju/outagefs","owner":"quark-zju","description":"fuse filesystem to emulate and test behaviors on power failures","archived":false,"fork":false,"pushed_at":"2020-08-25T17:03:29.000Z","size":69,"stargazers_count":8,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-07-09T00:58:03.903Z","etag":null,"topics":["fuse","testing"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/quark-zju.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-08-23T22:35:28.000Z","updated_at":"2025-04-03T15:35:09.000Z","dependencies_parsed_at":"2022-09-12T06:11:37.501Z","dependency_job_id":null,"html_url":"https://github.com/quark-zju/outagefs","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/quark-zju/outagefs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quark-zju%2Foutagefs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quark-zju%2Foutagefs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quark-zju%2Foutagefs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quark-zju%2Foutagefs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/quark-zju","download_url":"https://codeload.github.com/quark-zju/outagefs/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quark-zju%2Foutagefs/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270657642,"owners_count":24623461,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-15T02:00:12.559Z","response_time":110,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fuse","testing"],"created_at":"2024-11-10T21:15:17.073Z","updated_at":"2025-08-16T10:49:40.696Z","avatar_url":"https://github.com/quark-zju.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"outagefs\n========\n\n`outagefs` emulates power outage to test application and filesystem behaviors.\n\nIt works by recording filesystem changes at the block device level, and\nreplaying writes with unsynchronized writes dropped randomly. The recording is\ndone by using fuse to expose a monitored file that represents the block device.\n\nCurrently, `outagefs` is mainly developed and tested on Linux.\n\nInstallation\n------------\n\n`outagefs` can be installed via `cargo`:\n\n```bash\ncargo install outagefs\n```\n\nExample: \"atomic rename\" on ext4\n--------------------------------\n\nIt's common to create a file and rename it to overwrite an existing file, and\nexpect the file to either have the new content, or the old content. How does\nthat work practically on ext4? Let's find out.\n\n### Setup\n\nFirst, prepare a base image of ext4 with a file `b` in it having some content:\n\n```bash\n# or, try `-s 1m` and see if it makes a difference\ntruncate -s 3m base\n# or, try 'ext2'\nmkfs.ext4 base\nmkdir ext4root\nsudo mount -o loop -t ext4 base ext4root\nsudo sh -c 'seq 4000 \u003e ext4root/b'\nsudo umount ext4root\n```\n\n### Record\n\nThen, use outagefs to record the write + rename operation:\n\n```bash\n# try adding 'sync' before 'mv' if 'ext2' is used\noutagefs mount --record --sudo --exec 'mount -o loop -t ext4 $1 ext4root; seq 2 6000 \u003e ext4root/a; mv ext4root/{a,b}; umount ext4root'\n```\n\n(If the command failed with \"fusermount: option allow_other only allowed ...\",\nedit /etc/fuse.conf and uncommit `user_allow_other`, or run the `outagefs`\ncommand under root)\n\nThe above command uses `base` as the base image, mounts it as a single file with\nrecording turned on, and passes that single file as `$1` to the shell script. The\nshell script mounts the file as ext4 and makes changes to the ext4 filesystem.\nWriting to the mounted ext4 filesystem gets translated to low-level write and\nsync operations to the `$1` file. The `--record` flag tells `outagefs` to write\nthe changes back to disk as `changes`.\n\nLet's check that outagefs does record some changes:\n\n```bash\noutagefs show\n```\n\n### Verify\n\nThe property we want to verify is \"b should have either new or old content\".\nLet's express that in a script and name it `verify.py`:\n\n```python\nimport pathlib\npath = pathlib.Path(\"./ext4root/b\")\n\ndef seq(start, end):\n    return b\"\".join([b\"%d\\n\" % i for i in range(start, end + 1)])\n\ntry:\n    if not path.exists():\n        print(\"BAD: does not exist\")\n    else:\n        data = path.read_bytes()\n        if data == b\"\":\n            print(\"BAD: empty file\")\n        elif data == seq(1, 4000):\n            print(\"GOOD: old content\")\n        elif data == seq(2, 6000):\n            print(\"GOOD: new content\")\n        else:\n            print(\"BAD: unexpected content\")\nexcept Exception as ex:\n    print(f\"ERROR: {ex}\")\n```\n\nVerify the end state is good:\n\n```bash\noutagefs mount --sudo --exec 'mount -o loop -t ext4 $1 ext4root \u0026\u0026 python3 verify.py; umount ext4root'\n# should print 'GOOD: new content'\n```\n\nIt's also good if all writes are discarded:\n\n```bash\noutagefs mount --filter 0 --sudo --exec 'mount -o loop -t ext4 $1 ext4root \u0026\u0026 python3 verify.py; umount ext4root'\n# should print 'GOOD: old content'\n```\n\n### Generating Tests\n\nMore interesting tests will be when some writes are discarded while others\naren't.  In theory it's possible to look at `outagefs show` result and find out\nwhat to discard, and figure out bits as a \"filter\" (`1`: take, `0` or not\nmentioned: discard), and test it like:\n\n```bash\noutagefs mount --filter 1000000001000000011 --sudo --exec 'mount -o loop -t ext4 $1 ext4root \u0026\u0026 python3 verify.py; umount ext4root'\n```\n\nIt is time consuming to figure out interesting test cases manually.\n`outagefs` provides a subcommand to generate test cases:\n\n```bash\noutagefs gen-tests\n```\n\nThis will print strings in the `offset:bits` form, suitable for `--filter`.\n`gen-tests` respects `Sync` operations. If a `Sync` is not discarded, none of\nthe `Write`s before it would be discarded. It will also try to make the number\nof test cases bounded so tests can complete.\n\nNow, let's just use the generated tests and run the verify script on them:\n\n```bash\nfor f in $(outagefs gen-tests); do\n    outagefs mount --filter $f --sudo --exec 'mount -o loop -t ext4 $1 ext4root \u0026\u0026 python3 verify.py; umount ext4root'\ndone\n```\n\nTips\n----\n\n### More Challenging Tests\n\nThe tests above might be not challenging enough. For example, individual writes\nare atomic and Sync are expected to work as expected. Hardware might have\ndifferent properties. For example, having hardward-specific 2KB block size,\nor does not always respect Sync, or might corrupt data during writes.\nTo make it easier to exercise such behaviors, `outagefs` has a `mutate`\nsub-command to rewrite changes:\n\n```bash\noutagefs mutate --split-write --zero-fill --drop-sync\n```\n\nThe `changes` file will be updated with the rewritten result.  Note that the\ninternal filesystem state can break more easily. It's likely to see some tests\nerroring out at the `mount` command. It's also easier to trigger some errors\nlike `EUCLEAN` or hangs.\n\n\n### Convenient Way to Run Tests\n\nIt is verbose and error-prone to setup, record, and run tests manually.\nThe `run-suite` subcommand can be use to make it easier:\n\n```bash\noutagefs run-suite --sudo suite-examples/rename-no-fsync-ext2.py\n```\n\nThe above command will create a temporary directory, call the script with\n`prepare` to create the `base` image, then `changes` to make changes to record,\nand eventually `verify` to verify test cases. After testing, the temporary\ndirectory is deleted.\n\n\n### Bisecting Tests\n\nFor non-trivial changes, there are a lot of test cases. Most of the cases are\nnot very interesting. Only those that are transisting from a valid old state\nto a valid new state are:\n\n```\nTest Cases: |-------------|-------------|-----------|\nState:      | Old State   | Interesting | New State |\n```\n\nIt is more efficient to bisect the \"Interesting\" cases to find out obviously\nbroken cases. The verification script can choose to return exit code in the\n10 to 19 range to indicate states. Something like:\n\n```python\nif is_good_old_state():\n    sys.exit(11)\nelif is_good_new_state():\n    sys.exit(12)\n```\n\nThe `run-suite` command can use the information to bisect the test cases.\nIf there is nothing to bisect, `run-suite` will run the remaining tests in\norder.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fquark-zju%2Foutagefs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fquark-zju%2Foutagefs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fquark-zju%2Foutagefs/lists"}