{"id":20009796,"url":"https://github.com/plsyssec/vera","last_synced_at":"2026-02-25T07:14:46.076Z","repository":{"id":45117514,"uuid":"209374424","full_name":"PLSysSec/vera","owner":"PLSysSec","description":null,"archived":false,"fork":false,"pushed_at":"2022-01-07T05:08:12.000Z","size":12437,"stargazers_count":12,"open_issues_count":0,"forks_count":3,"subscribers_count":10,"default_branch":"ae","last_synced_at":"2025-05-04T19:39:57.971Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PLSysSec.png","metadata":{"files":{"readme":"README.md","changelog":"ChangeLog.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-09-18T18:10:56.000Z","updated_at":"2025-04-05T06:16:38.000Z","dependencies_parsed_at":"2022-09-17T14:10:34.200Z","dependency_job_id":null,"html_url":"https://github.com/PLSysSec/vera","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/PLSysSec/vera","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PLSysSec%2Fvera","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PLSysSec%2Fvera/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PLSysSec%2Fvera/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PLSysSec%2Fvera/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PLSysSec","download_url":"https://codeload.github.com/PLSysSec/vera/tar.gz/refs/heads/ae","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PLSysSec%2Fvera/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29813739,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-25T05:36:42.804Z","status":"ssl_error","status_checked_at":"2026-02-25T05:36:31.934Z","response_time":61,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-13T07:17:05.815Z","updated_at":"2026-02-25T07:14:46.046Z","avatar_url":"https://github.com/PLSysSec.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# VeRA code \n\nWelcome to the submission version of the VeRA code/artifact!\n\nPaper is [here](https://www.cs.utexas.edu/~hovav/dist/vera.pdf)\n\nArtifact will be here once it's posted\n\n## Install dependencies\n\n- z3\n- The Haskell tool [Stack](https://docs.haskellstack.org/en/stable/README/)\n\n## Build project\n\nOne you have all the dependencies installed, you can build the project:\n```\nstack build\n```\n\n## Run verification\n\nTo try to verify all the range analysis routines, run:\n```\nstack test --ta '-p Verification'\n```\n\nThis will take a long time (overnight-ish)!\n\n## Code map\n\nSource:\n- ActiveCode: Generate QuickCheck tests for C++/JS semantics\n- DSL: Implementation of C++ and JavaScript semantics\n- Generate: Generate SMT from C++\n- IonMonkeyGenerated:\n  - code.cpp: Firefox's range analysis routines in VeRA C++\n  - Operations: Connecting the VeRA C++ implementations to Haskell so we can verify them\n  - Verify: Verification code\n\n# VeRA Artifact Evaluation (instructions from the AE submission)\n\nHello, and thanks for evaluating this artifact!\n\nOur paper evaluation (Section 6) has six parts:\n\n1. Can VeRA prove Firefox range analysis correctness?\n2. Can VeRA proofs catch real correctness bugs?\n3. Are the VeRA proofs correct?\n4. Do the verified routines work correctly in Firefox?\n5. How do the verified routines perform in Firefox?\n6. How hard is it to integrate the verified routines into Firefox?\n\nWe are able to reproduce the results from points 1-5 (the sixth is a bit more subjective).\nNote that all timing results may vary, as will the counterexamples the solver chooses to\ndisplay for buggy routines.\n\n**WARNING: Expect to need around 110 GB of free disk space to run this evaluation.**\nWe're sorry, it contains multiple versions of the Firefox browser.\n\n## Setup\n\n1. Install virtualbox from virtualbox.org\n\n2. Use the 'import appliance' option from the file menu in virtualbox to import\n   vera.ova. You can either accept or modify the default options. \n\n3. Push the start button at the top of the screen to start the vm.\n\n4a. For a graphical interface, a login screen will pop up after starting the vm.\n    Login with username: vera and password: vera_user\n\n4b. For a non-graphical interface, you can connect to port 22222 on localhost: \n    ssh -p 22222 vera@127.0.0.1 with password vera_user \n\n5. Navigate to the ~/lejit/results directory \n\n## Clean the results directory\n\nFrom the **results** directory, run `python2 clean.py`. This will get rid of all results\nand all intermediate files (there shouldn't be any to begin with, but you can use this\nanytime to clean up the directory).\n\n## Getting started\n\nMake sure you are in the **results** directory! All python scripts should be run with\npython2. \n\nRun: From the **results** directory, `python2 sanity.py`\n\nExpected time: under a minute\n\nExpected result: verify_sanity.txt has a bunch of output followed by\n**1 out of 16 tests failed**.\n\nThis does a single run of a number of tests and verification routines. If the output\ndiffers from the expected output, please indicate so on HotCRP so we can quickly fix it. \n\n## Generate results for the entire paper at once\n\nMake sure you are in the **results** directory!\n\nFrom the **results** directory, type `python2 repro_all.py` to reproduce all\nresults in the paper.\n\nExpected time: Overnight\n\nThe results will all be generated in the results directory, in the following order:\n1. Can VeRA prove Firefox range analysis correctness?\n   Compare **results/verify_table.pdf** to Figure 8 in the paper.\n2. Can VeRA proofs catch real correctness bugs?\n   Compare **results/bug_examples.txt** to 6.1's **An old Firefox bug** and\n   **A new Firefox bug** bug examples. \n3. Are the VeRA proofs correct?\n   Compare **results/quickcheck.txt** to 6.1's **Are VeRA proofs correct?**.\n   Note that by default, our script runs quickcheck only 100 times per operator,\n   while for the paper we ran 1,000 times per operator. We provide an optional\n   way of running 1,000 quickcheck tests per operator below.\n4. Do the verified routines work correctly in Firefox?\n   Compare **results/firefox-js-tests.txt** to 6.2's **Do the verified routines\n   work correctly?**.\n5. How do the verified routines perform in Firefox?\n   Compare **results/jetstream2.pdf** to 6.2's **How do the verified routines perform?**.\n   Note that we do NOT re-run the latency experiments for 6.2, since these are graphical\n   tests and our artifact is a virtual machine with limited graphics capabilities.\n   You can re-generate the graphs from our original data, though (see point 5 below).\n\nAlternatively, to generate results for each claim individually, use the\nfollowing instructions:\n\n### (1) Can VeRA prove Firefox range analysis correctness?\n\nRun: make_verif_table.py\n\nLook at: results/verify_table.pdf\n\nCompare to: Figure 8\n\nExpected time: Overnight\n\nThe script for generating Figure 8 is called make_verif_table.py This\nscript uses command `stack test --ta '-p Verification',` which verifies\neach range analysis operator correct wrt to JavaScript semantics with\na timeout of 20 minutes. It will generate a standalone PDF of the time\nit took each verification condition to verify in verify_table.pdf\n\n### (2) Can VeRA proofs catch real correctness bugs?\n\nRun: generate_bugs.py\n\nLook at: results/bug_examples.txt\n\nCompare to: 6.1's **An old Firefox bug** and **A new Firefox bug** bug examples\n\nExpected time: 2-3 minutes \n\nThe script for generating the examples in 6.1's \"A new Firefox bug\" and\n\"An old Firefox bug\" are in generate_bugs.py This script uses command\n`stack test --ta '-p \u003ctest\u003e'`, where `test` is brokenIntersectTest or\nbrokenCeilTest. This runs verification rountines for\neither buggy operator and displays (1) a counterexample showing that each\noperator is buggy and (2) the time it took to generate that example. The\noutput will be in bug_examples.txt\n\n### (3) Are the VeRA proofs correct?\n\nRun: quickcheck.py\n\nLook at: quickcheck.txt\n\nCompare to: 6.1's **Are VeRA proofs correct?**\n\nExpected time: ~30 minutes \n\nThe script for generating the quickcheck tests in 6.1 is in quickcheck.py.\nIt runs the command `stack test --ta -p JS_Fast/Cpp_Fast`. \nBy default, the script runs **100** random tests for each JS or C++ operator.\nIn the paper, we run quickcheck tests 1,000 times per operator---we do not\ndo so for time reasons in the artifact eval. If you would like to run quickcheck\nfor longer, you can use quickcheck_long.py in the same way as quickcheck.py. \nStill, it will not be the exact result from the paper, since each run of quickcheck\nproduces new random tests.\n\n### (4) Do the verified routines work correctly in Firefox?\n\nRun: firefox-js-tests.py\n\nLook at: firefox-js-tests.txt\n\nCompare to: 6.2's **Do the verified routines work correctly?**\n\nExpected time: ~30 minutes\n\nFirefox has 3 main test suites for Javascript and the JIT: `jstests`,\n`jsapi-tests`, and `jit-test`. The script runs all three and logs their\nresults.\n\n### (5) How do the verified routines perform in Firefox?\n\nRun: jetstream2.py\n\nLook at: `jetstream2.pdf`\n\nCompare to: 6.2's **How do the verified routines perform?**\n\nExpected time: ~30 minutes \n\nThis script executes the JetStream2 benchmark suite for both versions of\nFirefox and produces an itemized graph comparing the results. For display\npurposes, the individual benchmarks are sorted based on score. This may cause\nthe order to differ slightly from the paper, because these benchmarks have a\ntendency to be noisy. On the topic of noise, any large differences in performance\nbetween the two should steady out on further runs.\n\n** Note: We do not reproduce the latency numbers but the graph can be built **\n\nThe results of the latency test can be found in\n`~/proofmonkey-gecko-dev[-original]/testing/mozharness/build/local.json`, and\nthe graph can be built by running `firefox-latency.py` in the `results`\ndirectory.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fplsyssec%2Fvera","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fplsyssec%2Fvera","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fplsyssec%2Fvera/lists"}