{"id":31286847,"url":"https://github.com/sbu-fsl/iocov","last_synced_at":"2025-09-24T10:23:14.082Z","repository":{"id":313550714,"uuid":"607470553","full_name":"sbu-fsl/IOCov","owner":"sbu-fsl","description":"Framework to compute and analyze input and output coverage for file system testing","archived":false,"fork":false,"pushed_at":"2025-09-06T19:31:29.000Z","size":476,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-09-06T21:30:41.658Z","etag":null,"topics":["coverage","filesystem","testing"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sbu-fsl.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-02-28T03:00:59.000Z","updated_at":"2025-09-06T19:31:35.000Z","dependencies_parsed_at":"2025-09-06T21:30:43.365Z","dependency_job_id":"c46955f6-ef16-4ada-bb8d-6f59fcbccbb8","html_url":"https://github.com/sbu-fsl/IOCov","commit_stats":null,"previous_names":["sbu-fsl/iocov"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/sbu-fsl/IOCov","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sbu-fsl%2FIOCov","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sbu-fsl%2FIOCov/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sbu-fsl%2FIOCov/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sbu-fsl%2FIOCov/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sbu-fsl","download_url":"https://codeload.github.com/sbu-fsl/IOCov/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sbu-fsl%2FIOCov/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":276730442,"owners_count":25694454,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-24T02:00:09.776Z","response_time":97,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["coverage","filesystem","testing"],"created_at":"2025-09-24T10:23:08.782Z","updated_at":"2025-09-24T10:23:14.027Z","avatar_url":"https://github.com/sbu-fsl.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# IOCov: a framework to compute input and output coverage for file system testing\n\n## Introduction\n\nThis repository contains the source code for the ***IOCov*** tool, which appear \nin the paper **\"[Enhanced File System Testing through Input and Output Coverage](https://www.fsl.cs.stonybrook.edu/docs/mcfs/systor25cmiocov.pdf)\" (SYSTOR '25)**\nand the paper **\"[Input and Output Coverage Needed in File System Testing](https://www.fsl.cs.stonybrook.edu/docs/mcfs/iocov-hotstorage23.pdf)\" (HotStorage '23)**.\n\nIOCov is a set of scripts for end-to-end measurement of input and output coverage \nin file system testing tools. It relies on **LTTng** to trace system calls, then \nfilters, parses, and analyzes the trace logs to extract only the syscalls relevant \nto file system testing, while excluding unrelated noise (e.g., library loads or log \nwrites). This allows IOCov to compute input and output coverage accurately using \nonly operations that constitute actual test cases.  \n\nNote that the concept of *input and output coverage* is specific to file system \ntesting, where we employ specialized techniques to partition the I/O space and \ndefine coverage metrics. For more details, please refer to our \n[SYSTOR 2025 paper](https://www.fsl.cs.stonybrook.edu/docs/mcfs/systor25cmiocov.pdf).  \n\nWe have used IOCov to measure input/output coverage for multiple file system \ntesting tools including CrashMonkey, xfstests, Syzkaller, and \n[Metis](https://github.com/sbu-fsl/Metis), and identified under-tested and/or \nover-tested cases in many of them, which offer actionable insights for improvement \nand potentially more bug detection. IOCov can also be applied to other testing \ntools, but with a few requirements and caveats:  \n\n- IOCov works with **LTTng trace logs only**. Do not use other syscall trace logs \n  with IOCov. \n\n- IOCov is designed for **dynamic testing only**, i.e., executing file systems with \n  syscalls. It is not intended for static analysis that does not run the file system. \n\n- IOCov identifies pathnames of file system testing to keep only testing-related \n  traces. Therefore, testing tools must use **dedicated, recognizable test \n  directories**. Some tools have multiple directories, e.g., xfstests uses both \n  `/mnt/test` and `/mnt/scratch`. Check and edit the \n  `find_testing_filename` function in `src/utilities.py` for different tools. \n\n---\n\n## Associated Repositories\n\nThere are related repositories that demonstrate IOCov usage:\n\n- **[CM-IOCov](https://github.com/sbu-fsl/CM-IOCov)**  \n  Extends [CrashMonkey](https://github.com/utsaslab/crashmonkey) with an input \n  driver to improve input coverage, detecting more crash-consistency bugs. \n\n- **[Metis](https://github.com/sbu-fsl/Metis)**  \n  File system model-checking framework at implementation-level, featuring versatile \n  input coverage support and thorough input and state exploration. \n\n- **[RefFS](https://github.com/sbu-fsl/RefFS)**  \n  Fast and reliable user-space file system that supports model checking state \n  save/restore API for efficient file system model checking. \n\n---\n\n## Repository Structure\n\n- **`src/`**  \n  Main implementation of IOCov, including tracing, parsing, and coverage analysis. \n\n- **`hotstorage23/`**, **`FAST2024/`**, **`SYSTOR2025/`**, **`Conf-extension/`**  \n  Experimental setup, evaluation scripts, and artifacts from our publications. \n\n- **`kernel-analysis/`**  \n  Scripts for analyzing kernel system call behaviors. \n\n- **`MCFS/`**  \n  Files related to Metis (aka MCFS) model checking experiments. \n\n- **`Syzkaller/`**  \n  Integration code and scripts for running IOCov with the Syzkaller fuzzer. \n\n---\n\n## Citations\n\nYou are welcome to cite the following papers if you use IOCov in your work:\n\n```\n@INPROCEEDINGS{systor25cmiocov,\n  TITLE =        \"Enhanced File System Testing through Input and Output Coverage\",\n  AUTHOR =       \"Yifei Liu and Geoff Kuenning and Md. Kamal Parvez and Scott Smolka and Erez Zadok\",\n  BOOKTITLE =    \"Proceedings of the 18th ACM International Systems and Storage Conference (SYSTOR '25)\",\n  ADDRESS =      \"Virtual Event\",\n  MONTH =        \"September\",\n  YEAR =         \"2025\",\n  PUBLISHER =    \"ACM\",\n}\n```\n\n```\n@INPROCEEDINGS{fast24metis,\n  TITLE =        \"{Metis}: File System Model Checking via Versatile Input and State Exploration\",\n  AUTHOR =       \"Yifei Liu and Manish Adkar and Gerard Holzmann and Geoff Kuenning and Pei Liu and Scott Smolka and Wei Su and Erez Zadok\",\n  NOTE =         \"To appear\",\n  BOOKTITLE =    \"Proceedings of the 22nd USENIX Conference on File and Storage Technologies (FAST '24)\",\n  ADDRESS =      \"Santa Clara, CA\",\n  MONTH =        \"February\",\n  YEAR =         \"2024\",\n  PUBLISHER =    \"USENIX Association\"\n}\n```\n\n```\n@INPROCEEDINGS{hotstorage23iocov,\n  TITLE =        \"Input and Output Coverage Needed in File System Testing\",\n  AUTHOR =       \"Yifei Liu and Gautam Ahuja and Geoff Kuenning and Scott Smolka and Erez Zadok\",\n  BOOKTITLE =    \"Proceedings of the 15th ACM Workshop on Hot Topics in Storage and File Systems (HotStorage '23)\",\n  MONTH =        \"July\",\n  YEAR =         \"2023\",\n  PUBLISHER =    \"ACM\",\n  ADDRESS =      \"Boston, MA\"\n}\n```\n\n---\n\n## Contact\n\nFor any question, please feel free to contact Yifei Liu ([yifeliu@cs.stonybrook.edu](mailto:yifeliu@cs.stonybrook.edu))\nand Erez Zadok ([ezk@cs.stonybrook.edu](mailto:ezk@cs.stonybrook.edu)).\nAlternatively, you can create an issue in this repository.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsbu-fsl%2Fiocov","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsbu-fsl%2Fiocov","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsbu-fsl%2Fiocov/lists"}