{"id":31041013,"url":"https://github.com/icmccorm/mirilli","last_synced_at":"2025-09-14T09:02:58.481Z","repository":{"id":128228777,"uuid":"563887671","full_name":"icmccorm/mirilli","owner":"icmccorm","description":"A study of undefined behavior across foreign function boundaries in Rust libraries.","archived":false,"fork":false,"pushed_at":"2025-02-13T18:47:16.000Z","size":52924,"stargazers_count":3,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-23T14:42:20.250Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/icmccorm.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-11-09T14:48:19.000Z","updated_at":"2025-02-13T18:46:46.000Z","dependencies_parsed_at":null,"dependency_job_id":"f665c97c-9e10-4091-b818-4440ad9b2f9e","html_url":"https://github.com/icmccorm/mirilli","commit_stats":null,"previous_names":["icmccorm/mirilli"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/icmccorm/mirilli","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/icmccorm%2Fmirilli","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/icmccorm%2Fmirilli/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/icmccorm%2Fmirilli/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/icmccorm%2Fmirilli/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/icmccorm","download_url":"https://codeload.github.com/icmccorm/mirilli/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/icmccorm%2Fmirilli/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":275082376,"owners_count":25402339,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-14T02:00:10.474Z","response_time":75,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-09-14T09:01:54.997Z","updated_at":"2025-09-14T09:02:58.434Z","avatar_url":"https://github.com/icmccorm.png","language":"R","funding_links":[],"categories":["Static Checkers"],"sub_categories":[],"readme":"# Artifact - A Study of Undefined Behavior Across Foreign Function Boundaries in Rust Libraries\n\n## Purpose\nWe are applying for all three badges. Our dataset, tool, and compilation scripts are all publicly **Available** on Zenodo. By following this script, you will confirm that our tool and data processing scripts are **Functional** and **Reusable** by reproducing the statistics in our paper and replicating our entire data collection process for a subset of the bugs that we discovered.\n\n## Provenance\nAll materials relevant to this project are published on [Zenodo](https://doi.org/10.5281/zenodo.12727039) within an x86 Docker Image and in raw, uncompiled source.\n\nA preprint of our paper is available at on [arXiv](https://arxiv.org/abs/2404.11671).\n\n## Data\nThe artifact contains six files:\n\n* `README.md` - This README file.\n\n* `USAGE.md` - A guide on how to use and extend MiriLLI\n\n* `DATASET.md` - Documentation on the contents of our dataset.\n\n* `src.tar.gz` - the raw source code for our tool and data compliation scripts.\n\n* `data.tar.gz` -  the raw output from our data collection steps described in Section 3 of our paper. \n\n* `crates.tar.gz` - the contents of the [crates.io](https://crates.io) database on 9/20/2023.\n\n* `appendix.pdf` - the Appendix of our paper. \n\n* `preprint.pdf` - A preprint of our paper.\n\n* `x86-docker-image.tar.gz` - an x86 Docker image containing a working build of our tool.\n\nYou will need `appendix.pdf`, `preprint.pdf`, `DATASET.md`, `USAGE.md`, and `x86-docker-image.tar.gz` for the evaluation.\n\n### Data - Contents\nHere we provide a brief overview of the contents of our docker image (excluding configuration files, `.renv` files, `.gitignore`, `Dockerfile`, `makefile`, etc.)\n```\n├── DATASET.md          // Documentation for our dataset and data compilation scripts.\n├── USAGE.md            // Documentation on how to use and extend MiriLLI         \n├── data.tar.gz         // The (compressed) dataset \n├── appendix.pdf        // The appendix\n├── appendix            // LaTeX source for the appendix\n├── rust-install        // Our custom Rust toolchain\n├── rllvm-as            // A submodule containing a script for assembling LLVM bitcode\n├── scripts             // R and Bash scripts for data collection and processing.\n└── src                 // Source for early and late linting passes, which detected FFI bindings.\n```\nOur data collection took place over three stages using a variety of file formats and intermediate processing steps. We created comprehensive documentation for our dataset. For brevity, instead of including all 10 pages of documentation here, we provide it within `dataset/DATASET.md`. As part of evaluating the reusability of our tool, we will ask you to confirm that this documentation exists and answer a question about one part it. Here, though, we provide a brief summary its contents.\n```\n├── README.md           // Detailed documentation for the contents of each data collection step\n├── bugs.csv            // A list of every bug we found and reported in our evaluation\n├── bugs_excluded.csv   // Additional bugs that we found, but that had already been fixed, or are otherwise excluded\n├── exclude.csv         // Crates that were excluded from out evaluation due to OOM errors during compilation\n├── population.csv      // Every valid crate within the database. \n├── stage1              // Results from compiling every crate to find test cases with LLVM bitcode\n├── stage2              // Results from running every test case in Miri to find which ones executed the FFI\n└── stage3              // Results from running MiriLLI on each test case.\n```\n\n## Setup\n\nTo complete our evaluation, your system must meet the following requirements:\n* An x86 System with [Docker](https://www.docker.com/) installed.\n* 16 GB of RAM\n* 100 GB of free space\n\nWe have set the resource requirements to be more than strictly necessary to ensure that you will not encounter any issues while evaluating the artifact.\n\nTo complete our evaluation, you will need:\n1. A preprint of our paper\n2. the Appendix\n3. Our Docker image - `x86-docker-image.tar.gz`\n\nFollow these instructions to ensure that you have access to these components. \n\nFirst, download the files `x86-docker-image.tar.gz` and `appendix.pdf` from [Zenodo](https://doi.org/10.5281/zenodo.12727039), and download our paper from [arXiv](https://arxiv.org/abs/2404.11671). \n\nNow, you can import our Docker image. This command should take ~10 minutes to complete.\n```\n# docker load -i x86-docker-image.tar.gz\n```\nTo confirm that the image is functional, launch a new container and attach a shell. \n```\n# docker run -it mirilli /bin/bash\n```\nConfirm that you have access to our custom Rust toolchain, `mirilli`, by executing the following command.\n```\n# rustup toolchain list\n```\nYou should see the following output.\n```\nstable-x86_64-unknown-linux-gnu\nnightly-2023-09-25-x86_64-unknown-linux-gnu\nnightly-x86_64-unknown-linux-gnu\nmirilli (default)\n```\nIf all of these steps have been completed successfully, then you are ready to begin evaluating the artifact.\n\n## Usage\nComplete each of the following steps to evaluate our artifact. This assumes that you have successfully completed each of the steps shown in the previous section (**Setup**). Except for Step #1, all steps must be completed inside a Docker container launched from our image.\n\n### Overview\n* **Part 1** - *Check the Appendix* (5 human-minutes)\n* **Part 2** - *Validate Results and Examples in the Paper* (50 human minutes + 20 compute minutes)\n* **Part 3** - *Replicate Data Collection Steps* (20 human minutes, 45 compute minutes)\n* **Part 4** - *How to Reuse Beyond the Paper* (10 human-minutes, 5 compute minutes)\n\nAs part of several steps, we will prompt you to input console commands and check that their output matches what is shown in this document. Commands are prefixed with a \"#\".\n\n## Part 1 - Check the Appendix (5 human-minutes)\n\nIn this step, you will examine our Appendix to confirm that each of the sections we reference in our paper are present.\n\nIn Section III.B.a of our paper, we state the following:\n\n\u003e We provide a formal model of our value conversion functions in Section 2 of the Appendix.\n\n**Task:** Open the Appendix and confirm that we provide this model by navigating to any of the subsections withiun Section 2.\n\nAt the end of the introduction to Section IV of our paper (Results) and immediately prior to Section IV.A, we state the following:\n\n\u003e We refer to each bug using a unique numerical ID corresponding to tables in Section 1 of the Appendix.\n\n**Task:** Open the Appendix and navigate to Section 1, Table 2. Check to confirm that:\n* There are 46 Bug IDs\n* For every Bug ID, we include at least one link in at least one of the columns \"Issues\", \"Pull(s)\", or \"Commits\". \n\nYou have now completed this step of our evaluation.\n\n## Part 2 - Validate Results and Examples in the Paper (50 human minutes + 10 compute minutes)\nThe complete dataset is provided as part of our Docker image within the directory `dataset`. It is documented in the file `DATASET.md`.\nIn this stage, you will compile the dataset, examine its output, and compare it against several of the statistics shown in the paper to confirm that \nthey can be replicated using our tool. This and each subsequent step requires our Docker image.\n\nAfter launching a container, execute the following command to build our dataset. **This command will take 3-5 minutes to complete**\n```\n# DATASET=dataset make build\n```\nThis will compile its contents from the `dataset` folder into the `build` folder. \n\nConfirm that this step has succeeded by executing the following command:\n\n```\n# tree -L 1 build/\n```\n\nYou should see the following output.\n\n```\n├── stage1\n├── stage2\n├── stage3\n├── stats.csv\n├── stats_long.csv\n└── visuals\n```\nTo complete this step, you will be using the files `build/stats_long.csv` and `build/visuals/bug_counts_table.csv`.\n\n### Part 2.1 - Inline Statistics - (20 human minutes, 5 compute minutes)\nTo validate this section, you will need to have a copy of the paper, this README, and the contents of the file `build/stats_long.csv`. This file contains two columns: `key` and `value`. Each value is a statistic, and each \nkey is an identifier that links to both a table in DATASET.md and the CSV file `dataset/stat_key_descriptions.csv`. Each of these files describe the meaning of each statistic. \n\nHere, you will reproduce a subset of our inline statistics to confirm that they can be replicated from our dataset. \n\nNavigate to Section III.A of the paper (\"Sampling\") on page 6. Skim this section and find at least 1-3 of the quotes in the column \"Quotes\" of the table shown below. When you find a quote, look in the table for its corresponding \"Key\". Then, execute the following command with `[key]` replaced by the string under \"Key\". Confirm that the number shown on that line matches the statistic shown in the text.\n```\ngrep -r \"[key],\" ./build/stats_long.csv\n```\n\n| Quote                                                   | Key |\n|---------------------------------------------------------|---|\n| It contained *125,804* unique crates                      | `num_crates_unfiltered`    |\n| (*121,015*) had at least one valid published version. | `num_crates_all`    |\n| (*84,106*) compiled without intervention.             | `num_crates_compiled`    |\n| (*44,661*) had unit tests.                            | `num_crates_had_tests`    |\n| (*11,120*) produced LLVM bitcode files                 | `num_crates_had_bytecode`    |\n| (*3,785*) of crates with both unit tests and bitcode   | `num_crates_had_tests_and_bytecode`    |\n| *88,637* tests that we identified                         | `test_count_overall`    |\n| (*47,189*) passed                                     | `tests_passed`    |\n| (*36,766*) failed                                     | `tests_failed`    |\n| (*3,869*) timed out                                    | `tests_timed_out`    |\n| (*1,178*) had been manually disabled                   | `tests_disabled`    |\n| (*23,116*) had failed due to foreign function calls.  | `tests_failed_ffi`    |\n| (*9,130*) called a foreign function we could execute. | `meta_llvmengaged`    |\n| (*957*) of the crates with tests and bytecode         | `meta_crates_llvmengaged`    |\n\nAll of the inline statistics in Section III and IV were taken from this file, with a few exceptions.\nAt the end of Section III.A, we report on the percentage of crates with foreign function bindings. \nWe collected these statistics by querying the database directly, using the output from building our\ndataset. We have excluded this section from our replication to save time and reduce the size of our Docker image, which is already substantial to support building and testing each of these components. We provides these queries in `src/scripts/dependents.sql`.\n\n### Part 2.2  - Section IV, Table I - (5 human minutes)\nNavigate to Table 1, which is at the top of Page #7.\nThis table was generated manually from the CSV file `build/visuals/bug_counts_table.csv`. \nThe layout of this file is a 1:1 match for the table.\nView its contents by executing the following command:\n```\n# cat build/visuals/bug_counts_table.csv\n```\nCompare the numbers with the counts shown in the table to confirm that they match. \n\n### Part 2.3 - Figure 3 - (10 human minutes, 1 compute minute)\nWe provide a working version of this minimal example in the directory `demo/figures/3`.\nTo replicate the bug, navigate to this directory:\n```\n# cd /usr/src/mirilli/demo/figure3/\n```\nYou can view the Rust source code for version of this example with the bug by executing the following command:\n```\n# cat src/bug.rs   \n```\nAnd the C source code with this command: \n```\n# cat src/main.c\n```\nView each file to confirm that—together—these files match the example shown in Figure 3 (with the exception of `open_f` instead of `open`) with the lines highlighted in red still present.\n\nThen, execute this example in MiriLLI\n```\n# cargo miri run -- bug\n```\nYou should see the following output, indicating that Miri detected a bug.\n```\nerror: Undefined Behavior: read access through \u003c4326\u003e at alloc2082[0x8] is forbidden\n  --\u003e src/bug.rs:23:9\n   |\n23 |         b.cache\n   |         ^^^^^^^ read access through \u003c4326\u003e at alloc2082[0x8] is forbidden\n   |\n...\n```\n\nNow, view the version of the example that contains the fix by executing the following command:\n```\n# cat src/fix.rs\n```\nConfirm that it matches the example shown in figure 3 with the lines highlighted in red replaced with the lines highlighted in green.\n\nThen, execute this example in MiriLLI\n```\n# cargo miri run -- fix \n```\nThis command should complete without an error.\n\n### Part 2.4 - Figure 4 - (10 human minutes, 1 compute minute)\nWe provide a working version of this minimal example in the directory `demo/figures/3`.\nTo replicate the bug, navigate to this directory:\n```\n# cd /usr/src/mirilli/demo/figure4/\n```\nYou can view the Rust source code for version of this example with the bug by executing the following command:\n```\n# cat src/bug.rs   \n```\nAnd the C source code with this command: \n```\n# cat src/main.c\n```\nView each file to confirm that—together—these files match the example shown in Figure 4 with the lines highlighted in red still present.\n\nThen, execute this example in MiriLLI\n```\n# cargo miri run -- bug\n```\nYou should see the following output, indicating that Miri detected a bug.\n```\n---- Foreign Error Trace ----\n\n@ %10 = load i32, ptr %9, align 8, !dbg !32\n\n/usr/src/mirilli/demo/figure4/src/main.c:24:46\nsrc/bug.rs:16:18: 16:48\n-----------------------------\n\nerror: Undefined Behavior: read access through \u003c4441\u003e at alloc2114[0x0] is forbidden\n...\n```\nNow, view the version of the example that contains the fix by executing the following command:\n```\n# cat src/fix.rs\n```\nConfirm that it matches the example shown in figure 4 with the lines highlighted in red replaced with the lines highlighted in green.\n\nThen, execute this example in MiriLLI\n```\n# cargo miri run -- fix \n```\nThis command should complete without an error.\n\n## Part 3 - Replicate Data Collection Steps - (20 human minutes, 45 compute minutes)\n\nWe evaluated our tool in three stages. \n\n* **Stage 1** - Find crates with unit tests that produce LLVM bitcode\n* **Stage 2** - Find tests from these crates that call foreign functions\n* **Stage 3** - Execute these tests in our custom dynamic analysis tool\n\nThe details of specific output files from each stage are documented in `DATASET.md`. Here, we focus on the describing the minimum requirements and necessary steps for finding bugs. \n\nFully replicating each of these steps for every published crate would take several days and hundreds of dollars in compute. To save you time, instead of running a full evaluation, you will replicate these stages for a subset of the crates where we found bugs. \n\nFor convenience, we provide a \"large\" and \"small\" subsets for replicating our steps. The \"small\" sample contains 3 of the 37 crates where we found bugs. We will use this sample to test our first and second stages of data collection. The \"large\" sample contains triggering test cases from 29 of the 37 crates where we found bugs. We exclude 8 crates from this sample because 7 no longer compile with this nightly version of the Rust toolchain, and one relies on a library that is installed as part of this docker image, so it no longer statically links by default. \n\nCollecting and parsing data requires creating a directory to hold intermediate results. This directory must contain a file `population.csv` with the columns `crate_name` and `version`, in that order. Each of `demo/large` and `demo/small` contains this file. \n\nTo begin, navigate to the root directory: \n```\n# cd /usr/src/mirilli\n```\nMake sure that the `build` directory has been deleted.\n```\n# rm -rf ./build\n```\nExecute the following command to view an example of the file `population.csv` for our `small` sample, which contains 3 crates.\n```\n# cat demo/small/population.csv\n```\nNote that in the actual dataset (`./dataset/population.csv`), this file contains each of the ~120,000 valid crates that were published at the time of writing. We parallelized this data collection process by splitting this CSV file into N partitions, with each partition executed on a separate machine.\n\n### Part 3.1 - Stage 1 - (5 human minutes, 5 compute minutes)\nIn this stage, we compiled every public Rust crate to find ones with test cases that produced LLVM bitcode.\n\nThe script for executing this stage is `./scripts/stage1/run.sh`. Execute the following command to view its purpose and requirements:\n```\n# ./scripts/stage1/run.sh\n```\nExecute the following command to begin data collection. **This will take about 1 minute to complete.**\n```\n# ./scripts/stage1/run.sh demo/small\n```\nThis will create the directory `demo/stage1`. Execute the following command to print its contents.\n```\n# tree demo/small/stage1 -L 1\n```\nYou should see the following output:\n```\ndemo/small/stage1\n├── bytecode\n├── early\n├── has_bytecode.csv\n├── late\n├── status_comp.csv\n├── status_download.csv\n├── status_lint.csv\n├── tests\n└── visited.csv\n\n4 directories, 5 files\n```\nCompile the raw data from Stage 1 using the following command:\n```\n# DATASET=demo/small make build/stage1\n```\nYou should see the following output:\n```\nStarting Stage 1...\nProcessing early lint results...\nProcessing late lint results...\nProcessing test results...\nFinished Stage 1\n```\nExecute the following command to confirm that this stage was successful.\n```\n# tree build/stage1\n```\nYou should see the following output:\n```\nbuild/stage1\n├── category_error_counts.csv\n├── early_abis.csv\n├── error_info.csv\n├── error_locations.csv\n├── finished_early.csv\n├── finished_late.csv\n├── had_ffi.csv\n├── has_tests.csv\n├── late_abis.csv\n├── lint_info.csv\n├── stage1.stats.csv\n└── stage2.csv\n\n0 directories, 12 files\n```\nHere, we're concerned with the file `stage2.csv`, which contains the list of crates that had unit tests and produced bytecode during their build process. Run the following command to validate that it contains the crate we tested.\n```\n# cat build/stage1/stage2.csv\n```\nYou should see the following output:\n```\nbzip2,0.4.4\ndec,0.4.8\nlibrsync,0.2.3\n```\nThis indicates that each of the 3 crates that we used as input to this step produced bytecode and had test cases that we can execute.\nThis file will be used as input to Stage 2.\n\n### Part 3.2 - Stage 2 - (5 human minutes, 10 compute minutes)\nIn this data collection stage, we ran every test for crates where we found bytecode in an unmodified version of Miri to find tests that called foreign functions. \n\nThe script for executing this stage is `./scripts/stage2/run.sh`. Execute the following command to view its purpose and requirements:\n```\n# ./scripts/stage2/run.sh\n```\nTo complete data collection for this stage, execute the following command. **This will take ~5 minutes to complete**\n```\n# ./scripts/stage2/run.sh demo/small ./build/stage1/stage2.csv\n```\nThis will compile and execute every test case found in the first stage. When running the script, you should have seen output like so:\n```\n...\nRunning read::tests::smoke3...\nExit code is 1\nMiri found FFI call for read::tests::smoke3\n...\nFINISHED!\n```\nThis will create the directory `demo/stage2`. Execute the following command to print its contents.\n```\n# tree demo/small/stage2 -L 1\n```\nIf this step succeeded, you should see the following output:\n```\ndemo/small/stage2\n├── info\n├── logs\n├── status_download.csv\n├── status_miri_comp.csv\n├── status_rustc_comp.csv\n├── tests.csv\n└── visited.csv\n\n2 directories, 5 files\n```\nExecute the following command to compile the dataset for this stage.\n```\n# DATASET=demo/small make ./build/stage2\n```\nYou should see the following output:\n```\nStarting Stage 2...\nFinished Stage 2\n```\nTo confirm that this stage is successful, execute the following command:\n```\n# tree build/stage2/\n```\nYou should see the following output (excluding the annotations)\n```\nbuild/stage2/\n├── stage2-ignored.csv\n├── stage2.stats.csv\n├── stage3.csv\n└── tests.csv\n\n0 directories, 4 files\n```\nThe file `stage3.csv` is typically used as input to Stage 3. It contains a list of each of the test cases that called foreign functions.\n\n### Part 3.3 - Stage 3 - (10 human minutes, 30 compute minutes)\nIn this stage, we used MiriLLI to execute each of the tests that we found in Stage 2. We had to complete this stage twice; once for each memory mode (as described in Section III). \n\nThe script for this stage is `./scripts/stage3/run.sh`. Execute it without arguments to see its description.\n```\n# ./scripts/stage3/run.sh\n```\nThe third argument, `-z`, is optional. If provided, then MiriLLI is executed in the \"zeroed\" memory mode, which\nzero-initializes all LLVM-allocated memory by default. We will only test the zeroed mode, since this is required\nfor replicating a subset of our bugs.\n\nInstead of using data from the previous stage, we will use a subset of the test cases where we found bugs. This consists of 31 tests from 29 crates. \nWe updated the underlying Rust toolchain that MiriLLI depends on to version 1.81.0 after we completed our evaluation, and a few crates no longer compile with this version. A few tests triggered multiple bugs—after we fixed one, another appeared—so we only include them once here. The test case for Bug #19 is no longer replicable, but Bug #20 is still replicable, and it is of the same nature from the same underlying library. We expect that this is due to a bug in our implementation. We will update our artifact if we find the root cause of this issue. We have documented each of these limitations in `dataset/bugs.csv`, \n\nExecute the following command to run the tests in zeroed mode. **This will take 20-30 minutes to complete**\n```\n# ./scripts/stage3/run.sh demo/large demo/large/stage3.csv -z\n```\nExecute the following command to view the output of this stage.\n```\n# tree demo/large/stage3/zeroed -L 1\n```\nYou should see the following output:\n```\ndemo/large/stage3/zeroed\n├── crates\n├── status_download.csv\n├── status_miri_comp.csv\n├── status_native_comp.csv\n├── status_native.csv\n├── status_stack.csv\n├── status_tree.csv\n└── visited.csv\n```\n\nCopy the output from this execution to an \"uninit\" directory, as if we had run that evaluation mode.\n```\n# cp -r demo/large/stage3/zeroed demo/large/stage3/uninit \n```\nNow, compile the Stage 3 results with the following command:\n```\n# DATASET=demo/large make ./build/stage3\n```\nYou should see the following output:\n```\nStarting Stage 3...\nProcessing errors from 'zeroed' mode...\nProcessing errors from 'uninit' mode...\nFinished Stage 3\n```\nConfirm that this stage was successful by executing the following command:\n```\n# tree ./build/stage3 -L 1\n```\nYou should see the following output:\n```\n./build/stage3\n├── diff_errors_uninit.csv    // errors that only occurred in uninit mode\n├── diff_errors_zeroed.csv    // errors that only occurred in zeroed mode\n├── errors.csv                // all errors (not-deduplicated)\n├── errors_unique.csv         // deduplicated errors\n├── failures.csv              // tests that failed, under either mode\n├── metadata.csv              // metadata flags set during run-time\n├── stage3.stats.csv          // in-text statistics\n├── uninit                    // Additional error information for each mode\n└── zeroed  \n```\nTo confirm that you have successfully reproduced our results, execute the following command:\n```\n# wc -l ./build/stage3/errors_unique.csv\n```\nYou should see the following output, indicating that there were 30 unique errors (with one additional line for the CSV header).\n```\n31 ./build/stage3/errors_unique.csv\n```\nExecute the following command to see a sample of our results for the crate `dec`.\n```\ngrep \"dec,\" ./build/stage3/errors_unique.csv\n```\nYou should see the following output:\n```\ndec,0.4.8,test_overloading,0,1,1,Using Uninitialized Memory...\ndec,0.4.8,test_decimal128_special_value_coefficient,0,1,1,Borrowing Violation...\n```\nFrom this point onward, we manually investigated the results in the files `errors_unique.csv` and `diff_errors[uninit/zeroed].csv`, recreating errors locally using MiriLLI and reporting them to maintainers. \n\nYou have now completed this stage of our evaluation.\n\n## Part 4 - How to Reuse Beyond the Paper (20 human-minutes, 5 compute minutes)\nThe guide in Part 4 can be used to replicate our evaluation on any set of crates.\n\nWe provide two additional files that document our tool and dataset to help future evaluators replicate our results and extend our tool. As previously mentioned, we document the contents and structure of our dataset in detail within the file `DATASET.md`. The file `USAGE.md` provides a brief introduction to our toolchain, as well as steps for building our Docker image. It describes the configuration options that we added to Miri, which support the memory and initialization modes we describe in Section III.B of our paper. This file also provides a guide to extending and maintaining MiriLLI with links to relevant areas of our source code for each key component.\n\nOur toolchain can still be used on recently published crates. The crate `bzip2` was at version 0.4.4 when we conducted our evaluation, but it has since been updated to verion 0.5.0, and ownership of the library has changed. However, the bug that we detected is still present. You can replicate it here, now, by following these steps. \n\nFirst, download the newest version of the library.\n```\n# cargo-download bzip2==0.5.0 -x -o bzip2 \n```\nThen, enter the directory and test it. Ensure that the current toolchain is set to `mirilli`.\n```\n# cd bzip2\n# rustup override set mirilli\n# cargo miri test -- bufread::tests::bug_61\n```\nYou should see the following output for the test `bufread::tests::bug_61`, indicating a cross-language aliasing violation.\n```\n---- Foreign Error Trace ----\n\n@ %250 = load i32, ptr %249, align 8, !dbg !379\n\n.../bzip2-sys-0.1.11+1.0.8/bzip2-1.0.8/decompress.c:197:178\n.../bzip2-1.0.8/bzlib.c:842:20\nsrc/mem.rs:236:19: 236:62\n-----------------------------\n\nerror: Undefined Behavior: attempting a read access using \u003c186391\u003e at alloc62307[0x8], but that tag does not exist in the borrow stack for this location\n```\nYou have now completed our artifact evaluation.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ficmccorm%2Fmirilli","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ficmccorm%2Fmirilli","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ficmccorm%2Fmirilli/lists"}