{"id":18498364,"url":"https://github.com/softdevteam/error_recovery_experiment","last_synced_at":"2026-07-03T15:30:54.171Z","repository":{"id":66490152,"uuid":"128940354","full_name":"softdevteam/error_recovery_experiment","owner":"softdevteam","description":null,"archived":false,"fork":false,"pushed_at":"2020-05-29T14:40:11.000Z","size":150,"stargazers_count":1,"open_issues_count":0,"forks_count":2,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-10-24T04:28:32.490Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/softdevteam.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-04-10T13:49:06.000Z","updated_at":"2024-10-09T01:14:14.000Z","dependencies_parsed_at":"2023-03-04T22:00:15.883Z","dependency_job_id":null,"html_url":"https://github.com/softdevteam/error_recovery_experiment","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/softdevteam/error_recovery_experiment","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/softdevteam%2Ferror_recovery_experiment","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/softdevteam%2Ferror_recovery_experiment/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/softdevteam%2Ferror_recovery_experiment/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/softdevteam%2Ferror_recovery_experiment/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/softdevteam","download_url":"https://codeload.github.com/softdevteam/error_recovery_experiment/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/softdevteam%2Ferror_recovery_experiment/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35091822,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-03T02:00:05.635Z","response_time":110,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-06T13:39:07.983Z","updated_at":"2026-07-03T15:30:54.129Z","avatar_url":"https://github.com/softdevteam.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Experiment for \"Don't Panic! Better, Fewer, Syntax Errors for LR Parsers\"\n\nThis is the experiment for the paper [\"Don't Panic! Better, Fewer, Syntax\nErrors for LR Parsers\"](https://arxiv.org/abs/1804.07133) by Lukas Diekmann and\nLaurence Tratt. The repository for the paper can be found at\nhttps://github.com/softdevteam/error_recovery_paper.\n\n\n## Obtaining a corpus\n\n### Using the same corpus we did\n\nYou will need to download a fixed release of the experiment from\nhttps://github.com/softdevteam/error_recovery_experiment that contains a\n`blackbox/combos` file. This allows you to recreate the corpus we used\nof real Java code from [Blackbox](https://bluej.org/blackbox/), part of the\nBlueJ editor. For confidentiality reasons, Blackbox data cannot be directly\ndistributed, but we can distribute the (inherently anonymous) database IDs\nwhich allow you to recreate precisely the corpus we used in the paper. In order\nto do this, you need to send an email to the Blackbox developers\n`blackbox-admin@bluej.org` requesting access. If/when your request is accepted,\nyou will be given access to the Blackbox server.\n\nLogin to the Blackbox server and execute the following commands:\n\n```\nmkdir error_recovery_experiment\ncd error_recovery_experiment\nwget https://archive.org/download/error_recovery_experiment/0.4/error_recovery_experiment-0.4.tar.gz\ntar xfz error_recovery_experiment-0.4.tar.gz\ncd blackbox\nmkdir src_files\n./combos_to_files.py\ntar cfz ~/src_files.tgz src_files\n```\n\nThe `combos_to_files.py` command takes about 9 hours to run, and the `tar`\ncommand a further 30 minutes. This will turn the IDs in `blackbox/combos` into\na compressed tar file of source files located at `~/src_files.tgz`. It is this\nlatter file (about 80MiB big) which you should then copy to the machine that\nyou wish to run the main experiment on. Note that this file contains data that\nis subject to your agreement with Blackbox (i.e. it must not be distributed to\nanyone who was not named in the initial Blackbox request).\n\nCopy the `src_files.tgz` file to the machine you wish to run the main\nexperiment on. **Please do not run the experiment on the Blackbox server!**\n\n\n### Generating a new Blackbox corpus\n\nYou can generate a new corpus of data from\n[Blackbox](https://bluej.org/blackbox/), part of the BlueJ editor. For\nconfidentiality reasons, Blackbox data cannot be directly distributed, but you\ncan obtain an agreement to access it yourself. In order to do this, you need to\nsend an email to the Blackbox developers `blackbox-admin@bluej.org` requesting\naccess. If/when your request is accepted, you will be given access to the\nBlackbox server.\n\nLogin to the Blackbox server and execute the following commands:\n\n```\ngit clone https://github.com/softdevteam/error_recovery_experiment\ncd error_recovery_experiment/blackbox\n```\n\nYou will then need to edit the `gen_combos.py` file at around line 50 to\nspecify the username and password that the BlueJ developers have given you.\n\nYou will then need to install a recent stable version of Rust (e.g. using\n[Rustup](https://rustup.rs/)) for your user.\n\nYou can then build the blackbox data:\n\n```\n./build_blackbox_data.sh\ntar cfz ~/src_files.tgz src_files\n```\n\nCopy the `src_files.tgz` file to the machine you wish to run the main\nexperiment on. **Please do not run the experiment on the Blackbox server!**\n\n\n## Installing dependencies\n\nTo run the experiments on your local machine, you need to install LaTeX, Python\n2.7, Rust 1.42 (or later). To build the paper you will also need to install Dia\nand Inkscape.\n\nIn order to process the results (i.e. generate PDF graphs and LaTeX tables),\nyou will need to install the following Python libraries: matplotlib, pandas,\nand seaborn. There are various ways to do this (e.g. OS packages; PIP), so\nplease use whichever mechanism you are most comfortable with.\n\nThere are several ways to install Rust, but [Rustup](https://rustup.rs/)\nis generally the easiest, as it installs it solely for your user account:\n\n```\ncurl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh\n```\n\nWhen asked, select `1) Proceed with installation (default)` to install.\n\n\n## Running the experiment\n\nWhen you have installed the dependencies, and assuming you copied\n`src_files.tgz` to the root of your home directory, execute the following\ncommands:\n\n```\ngit clone https://github.com/softdevteam/error_recovery_experiment\ncd error_recovery_experiment/blackbox\ntar xfz ~/src_files.tgz\ncd ../runner\n./build.sh\n```\n\nThis will download and build the various parser variants used in the experiment\n(taking about 10 minutes).\n\nAt this point you then need to decide whether you want to run the full\nexperiment or not. The full experiment takes about 15 days to run and about 10\nhours to process the generated data. You can reduce the execution time by\ncutting down the number of times that data is rerun / reprocessed: this will\nslightly reduce the quality of the subsequent data, although it should not\nsubstantially affect your ability to tell whether you have replicated the\npaper's results or not. We suggest changing: the `PEXECS` variable in\n`runner/run.py` from 30 to 3; and the `BOOTSTRAP` variable in\n`runner/process.py` from 10,000 to 100. With these numbers, the experiment\ntakes about 11 hours to run and data processing takes about 10 minutes to run.\n\nYou should then execute the following commands:\n\n```\n./run.sh\n./process.py\n```\n\nThe `run.sh` command produces some rough statistics as it progresses -- these\nare simple sanity checks that are not fully post-processed and are not directly\ncomparable with numbers in the paper. `process.py` produces a number of `.ltx`\nand `.pdf` files.  You can examine these individually, or you can rebuild our\npaper with the data from your run by first downloading the paper source:\n\n```\ngit clone https://github.com/softdevteam/error_recovery_paper\ncd error_recovery_paper\n```\n\nCopy the `.ltx` and `.pdf` files you have created into the root of the\n`error_recovery_paper` directory and execute `make` (if on *BSD, you will need\nto use `gmake`). This will build a file `error_recovery.pdf` which should be a\nversion of the paper with your data in.\n\n\n## A brief guide to the source code\n\nThe experiment downloads the\n[grmtools](https://github.com/softdevteam/grmtools/) library, checks out a\nfixed version of it, and then builds several variants of a Java parser (the\n`runner/java_parser_*` executables). It does this by repeatedly patching\ngrmtools and the Java parser, clearing any existing build, and then rebuilding\nthe Java parser. `runner/build.sh` shows which patches are used for each\nvariant.\n\nThe source code for the Java parser can be found in `runner/java_parser`: in\nessence, it is a thin wrapper around the Java grammar which reports some simple\nstatistics.\n\nFor more information on grmtools itself, the best starting point is the\n[grmtools book](https://softdevteam.github.io/grmtools/master/book/) and/or the\nAPI documentation (this is split into sub-libraries, though the [lrpar\nAPI](https://docs.rs/lrpar/0.6.2/lrpar/) is probably the most relevant).\n\nThe bulk of the error recovery code itself can be found in\n`runner/grmtools/lrpar/src/lib/cpctplus.rs` although some of the helper\nfunctions are in `runner/grmtools/lrpar/src/lib/mf.rs`. Both files are fairly\nheavily commented to help you match them up against the paper.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsoftdevteam%2Ferror_recovery_experiment","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsoftdevteam%2Ferror_recovery_experiment","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsoftdevteam%2Ferror_recovery_experiment/lists"}