{"id":20859387,"url":"https://github.com/engineeringsoftware/jattack","last_synced_at":"2025-05-12T08:32:28.765Z","repository":{"id":63528111,"uuid":"541269451","full_name":"EngineeringSoftware/jattack","owner":"EngineeringSoftware","description":"Compiler Testing using Template Java Programs (ASE'22 Distinguished Paper Award)","archived":false,"fork":false,"pushed_at":"2023-08-30T05:13:50.000Z","size":483,"stargazers_count":18,"open_issues_count":0,"forks_count":3,"subscribers_count":2,"default_branch":"master","last_synced_at":"2023-08-30T12:13:32.624Z","etag":null,"topics":["compiler","java","jit-compiler","jvm","template","testing"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/EngineeringSoftware.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-09-25T18:26:50.000Z","updated_at":"2023-07-30T13:09:12.000Z","dependencies_parsed_at":"2023-02-17T12:00:35.533Z","dependency_job_id":null,"html_url":"https://github.com/EngineeringSoftware/jattack","commit_stats":null,"previous_names":[],"tags_count":1,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EngineeringSoftware%2Fjattack","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EngineeringSoftware%2Fjattack/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EngineeringSoftware%2Fjattack/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EngineeringSoftware%2Fjattack/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/EngineeringSoftware","download_url":"https://codeload.github.com/EngineeringSoftware/jattack/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225130723,"owners_count":17425506,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["compiler","java","jit-compiler","jvm","template","testing"],"created_at":"2024-11-18T04:49:41.874Z","updated_at":"2024-11-18T04:49:42.456Z","avatar_url":"https://github.com/EngineeringSoftware.png","language":"Java","readme":"# JAttack\n\nJAttack is a framework that enables template-based testing for\ncompilers. Using JAttack, compiler developers can write templates in\nthe same language as the compiler they are testing (Java), enabling\nthem to leverage their domain knowledge to set up a code structure\nlikely to lead to compiler optimizations while leaving holes\nrepresenting expressions they want explored. JAttack executes\ntemplates, exploring possible expressions for holes and filling them\nin, generating programs to later be run on compilers. JAttack blends\nthe power of developers insights, who are providing templates, and\nrandom testing to detect critical bugs.\n\n## Table of contents\n\n1. [Demo](#Demo)\n2. [Requirements](#Requirements)\n3. [Install](#Install)\n4. [Use](#Use)\n5. [Docs](#Docs)\n6. [Hall of Fame](#Hall-of-Fame)\n7. [Citation](#Citation)\n8. [Contact](#Contact)\n\n## Demo\n\nThis demo reproduces a bug of OpenJDK jdk-11.0.8+10 C2 JIT compiler\nusing template `T.java`.\n\n1. Developers write a template program using JAttack's DSL fully\n   embedded in Java, for example, `T.java`.\n\n```java\nimport jattack.annotation.Entry;\nimport static jattack.Boom.*;\n\npublic class T {\n\n    static int s1;\n    static int s2;\n\n    @Entry\n    public static int m() {\n        int[] arr = { s1++, s2, 1, 2, intVal().eval() };\n        for (int i = 0; i \u003c arr.length; ++i) {\n            if (intIdOrIntArrAccessExp().eval() \u003c= s2\n                    || relation(intId(\"s2\"), intIdOrIntArrAccessExp(), LE).eval()) {\n                arr[i] \u0026= arithmetic(intId(), intArrAccessExp(), ADD, MUL).eval();\n            }\n        }\n        return s1 + s2;\n    }\n}\n```\n\n2. JAttack executes the given template to generate concrete Java\n   programs. For example, one generated program from the\n   template `T.java` can be `TGen1.java`.\n\n```java\nimport jattack.annotation.Entry;\nimport static jattack.Boom.*;\nimport org.csutil.checksum.WrappedChecksum;\n\npublic class TGen1 {\n\n    static int s1;\n    static int s2;\n\n    public static int m() {\n        int[] arr = { s1++, s2, 1, 2, -1170105035 };\n        for (int i = 0; i \u003c arr.length; ++i) {\n            if (i \u003c= s2 || (s2 \u003c= arr[2])) {\n                arr[i] \u0026= (s2 + arr[0]);\n            }\n        }\n        return s1 + s2;\n    }\n\n    public static long main0(String[] args) {\n        int N = 100000;\n        if (args.length \u003e 0) {\n            N = Math.min(Integer.parseInt(args[0]), N);\n        }\n        WrappedChecksum cs = new WrappedChecksum();\n        for (int i = 0; i \u003c N; ++i) {\n            try {\n                cs.update(m());\n            } catch (Throwable e) {\n                if (e instanceof jattack.exception.InvokedFromNotDriverException) {\n                    throw e;\n                }\n                cs.update(e.getClass().getName());\n            }\n        }\n        cs.updateStaticFieldsOfClass(TGen1.class);\n        return cs.getValue();\n    }\n\n    public static void main(String[] args) {\n        System.out.println(main0(args));\n    }\n}\n```\n\n3. JAttack runs every generated program across Java JIT compilers\n   under test. For example, running the generated program `TGen1.java`\n   crashes C2 in openjdk-11.0.8.\n\n```\n#\n# A fatal error has been detected by the Java Runtime Environment:\n#\n#  SIGSEGV (0xb) at pc=0x00007f55deedd845, pid=432431, tid=432442\n#\n# JRE version: OpenJDK Runtime Environment AdoptOpenJDK (11.0.8+10) (build 11.0.8+10)\n# Java VM: OpenJDK 64-Bit Server VM AdoptOpenJDK (11.0.8+10, mixed mode, tiered, compressed oops, g1 gc, linux-amd64)\n# Problematic frame:\n# V  [libjvm.so+0xd60845]  ok_to_convert(Node*, Node*)+0x15\n#\n# Core dump will be written. Default location: Core dumps may be processed with \"/usr/share/apport/apport -p%p -s%s -c%c -d%d -P%P -u%u -g%g -- %E\" (or dumping to /home/zzq/projects/jattack/core.432431)\n#\n# If you would like to submit a bug report, please visit:\n#   https://github.com/AdoptOpenJDK/openjdk-support/issues\n#\n```\n\nTo run the demo, please run `./demo.sh`. Sample output:\n```\nDownload JDK...\nBuild JAttack jar...\nInstall python packages...\n1..3\n[10:28:28E]su.__main__: bash: line 1: 2414756 Aborted                 (core dumped) /home/zzq/projects/jattack/.downloads/jdk-11.0.8+10/bin/java -cp /home/zzq/projects/jattack/tool/jattack-all.jar:/home/zzq/projects/jattack/.jattack/T/build -XX:TieredStopAtLevel=4 -XX:ErrorFile=/home/zzq/projects/jattack/.jattack/T/output/TGen1/he_err_pid%p.log -XX:ReplayDataFile=/home/zzq/projects/jattack/.jattack/T/output/TGen1/replay_pid%p.log TGen1 \u003e /home/zzq/projects/jattack/.jattack/T/output/TGen1/java_env0.txt 2\u003e /dev/null\n\nnot ok 1 - TGen1\n  ---\n  message: 'Found a potential crash bug'\n  data: CrashBugData(type=\u003cBugType.CRASH: 'crash'\u003e, crashed_java_envs=[JavaEnv(java_home=PosixPath('/home/zzq/projects/jattack/.downloads/jdk-11.0.8+10'), java_opts=['-XX:TieredStopAtLevel=4'])])\n  ...\nok 2 - TGen2\nok 3 - TGen3\n```\n\n## Requirements\n\n- Linux with GNU Bash (tested on Ubuntu 20.04)\n- JDK \u003e=11\n- Python 3.8\n\n## Install\n\n```bash\ncd tool\n./install.sh\n```\n\nThe `install.sh` script builds JAttack jar, installs python packages\nand creates an executable `jattack` in `tools`.\n\n## Use\n\n```bash\ncd tool\n./jattack --clz TEMPLATE_CLASS_NAME --n_gen NUM_OF_GENERATED_PROGRAMS \\\n    [--java_envs JAVA_ENVIRONMENTS_UNDER_TEST]\n    [--src TEMPLATE_SOURCE_PATH]\n    [--n_itrs NUM_OF_ITERATIONS_TO_TRIGGER_JIT]\n    [--seed RANDOM_SEED]\n```\n\n### Examples of Run Commands:\n\n- Provide only two required arguments `--clz` and `--n_gen`.\n\n  ```bash\n  ./tool/jattack --clz T --n_gen 3\n  ```\n\n  This command generates 3 programs from template `T.java` and uses\n  the 3 generated programs to test default java environments found in\n  `$JAVA_HOME` at level 4 and level 1, which are:\n  - `$JAVA_HOME/bin/java -XX:TieredStopAtLevel=4`\n  - `$JAVA_HOME/bin/java -XX:TieredStopAtLevel=1`\n\n- Specify java environments and associated java options to be tested\n  using `--java_envs`.\n\n  ```bash\n  ./tool/jattack --clz T --n_gen 3 \\\n      --java_envs \"[\\\n          [/home/zzq/opt/jdk-11.0.15,[-Xbatch,-Xcomp,-XX:-TieredCompilation]],\\\n          [/home/zzq/opt/jdk-17.0.3,[-Xbatch,-Xcomp,-XX:TieredStopAtLevel=1]],\\\n          [/home/zzq/opt/jdk-17.0.3,[]]]\"\n  ```\n\n  The `java_envs` argument is a list, which can also be appended using\n  `--java_envs+=`, for example, the command above can be rewritten as:\n  ```bash\n  ./tool/jattack --clz T --n_gen 3 \\\n      --java_envs [[/home/zzq/opt/jdk-11.0.15,[-Xbatch,-Xcomp,-XX:-TieredCompilation]]] \\\n      --java_envs+=[[/home/zzq/opt/jdk-17.0.3,[-Xbatch,-Xcomp,-XX:TieredStopAtLevel=1]]] \\\n      --java_envs+=[[/home/zzq/opt/jdk-17.0.3,[]]]\n  ```\n\n  The `java_envs` argument can also be given using a config file, i.e.,\n  ```bash\n  ./tool/jattack --config config.yaml --clz T --n_gen 3\n  ```\n  where `config.yaml` is:\n  ```yaml\n  # config.yaml\n  java_envs:\n    -\n      - /home/zzq/opt/jdk-11.0.15\n      - - -Xbatch\n        - -Xcomp\n        - -XX:-TieredCompilation\n    -\n      - /home/zzq/opt/jdk-17.0.3\n      - - -Xbatch\n        - -Xcomp\n        - -XX:TieredStopAtLevel=1\n    -\n      - /home/zzq/opt/jdk-17.0.3\n      - []\n  ```\n\n  This command generates 3 programs from template `T.java` and uses\n  the 3 generated programs to test given java environments with given\n  options, which are\n  - `/home/zzq/opt/jdk-11.0.15/bin/java -Xbatch -Xcomp -XX:-TieredCompilation`\n  - `/home/zzq/opt/jdk-17.0.3/bin/java -Xbatch -Xcomp -XX:TieredStopAtLevel=1`\n  - `/home/zzq/opt/jdk-17.0.3/bin/java`\n\n### Full List of Arguments\n```\n  -h, --help            Show this help message and exit.\n  --config CONFIG       Path to a configuration file.\n  --print_config[=flags]\n                        Print the configuration after applying all\n                        other arguments and exit. The optional flags\n                        are one or more keywords separated by comma\n                        which modify the output. The supported flags\n                        are: comments, skip_default, skip_null.\n\n  --clz CLZ             the fully qualified class name of the\n                        template, separated with \".\" (required, type:\n                        str)\n  --n_gen N_GEN         the total number of generated programs\n                        (required, type: int)\n  --src SRC             the path to the source file of the template.\n                        By default, `./{clz}.java` is used. (type:\n                        Optional[str], default: null)\n  --n_itrs N_ITRS       the number of iterations to trigger JIT (type:\n                        int, default: 100000)\n  --seed SEED           the random seed used by JAttack during\n                        generation, fix this to reproduce a previous\n                        generation. (type: Optional[int], default:\n                        null)\n  --java_envs JAVA_ENVS, --java_envs+ JAVA_ENVS\n                        the java environments to be differentially\n                        tested, which should be provided as a list of\n                        a tuple of java home string and a list of java\n                        option strings, e.g., `--java_envs=[[/home/zzq\n                        /opt/jdk-11.0.15,[-XX:TieredStopAtLevel=4]],[/\n                        home/zzq/opt/jdk-17.0.3,[-XX:TieredStopAtLevel\n                        =1]]]` means we want to differentially test\n                        java 11 at level 4 and java 17 at level 1.\n                        Note, the first java environment of the list\n                        will be used to compile the template and\n                        generated programs, which means the version of\n                        the first java environment has to be less than\n                        or equal to the remaining ones. Also, the\n                        first java environment is used to run JAttack\n                        itself, which means its version should be at\n                        least 11. By default, $JAVA_HOME in the system\n                        with level 4 and level 1 are used, i.e., `--ja\n                        va_envs=[[$JAVA_HOME,[-XX:TieredStopAtLevel=4]\n                        ],[$JAVA_HOME,[-XX:TieredStopAtLevel=1]]]`\n                        (type: Optional[List[Tuple[str, List[str]]]],\n                        default: null)\n```\n\n### Output\n\nJAttack's command-line output is in [TAP](https://testanything.org/)\nformat, so you can make it prettier using any TAP consumer, like\n[tapview](https://gitlab.com/esr/tapview):\n```\n$ ./tool/jattack --clz T --n_gen 3 --seed 42 \\\n    --java_envs \"[\\\n        [.downloads/jdk-11.0.8+10,[-XX:TieredStopAtLevel=4]],\\\n        [.downloads/jdk-11.0.8+10,[-XX:TieredStopAtLevel=1]]]\" \\\n    | tapview\nF..\nnot ok 1 - TGen1\n  ---\n  message: 'Found a potential crash bug'\n  data: CrashBugData(type=\u003cBugType.CRASH: 'crash'\u003e, crashed_java_envs=[JavaEnv(java_home=PosixPath('/home/zzq/projects/jattack/.downloads/jdk-11.0.8+10'), java_opts=['-XX:TieredStopAtLevel=4'])])\n  ...\n3 tests, 1 failures.\n```\n\nAfter the run, a hidden directory `.jattack` is created under\ncurrent working directory with the following structure:\n```\n.jattack\n    - logs # logs of runs\n      - 1679956565593918684.log\n    - T\n      - build # Java class files\n        - TGen3.class\n      - gen # Generated programs from the template\n        - Gen1\n          - TGen1.java\n        - Gen2\n          - TGen2.java\n        - Gen3\n          - TGen3.java\n      - output # Outputs of generated programs executed on different java environments\n        - Gen1\n          - he_err_pid693345.log # error data file of the crash\n          - java_env1.txt # Output from execution on java_envs[1]\n          - replay_pid693345.log # replay data file of the crash\n        - Gen2\n          - java_env0.txt # Output from execution on java_envs[0]\n          - java_env1.txt\n        - Gen3\n          - java_env0.txt\n          - java_env1.txt\n```\n\n## Docs\n\nThe following steps build javadoc for JAttack jar. Please refer to\nclass `Boom` for how to use provided APIs to write your own template.\n\n1. Build javadoc from source code.\n   ```bash\n   cd tool/api\n   ./gradlew javadoc\n   ```\n\n2. Open `tool/api/build/docs/javadoc/index.html` in your favorite\n   browser.\n\n## Hall of Fame\n\nDirectory `bugs` contains all the JIT bugs we found using JAttack,\neach of which contains a template program, a generated program and a\nminimized program to expose the bug.\n\nIf you find JIT bugs using JAttack, we would be happy to add your\nfindings to this list. Please open a PR with a link to your bug.\n\n- [JDK-8239244](https://bugs.openjdk.java.net/browse/JDK-8239244)\n  (Login required): See\n  [CVE-2020-14792](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-14792)\n- [JDK-8258981](https://bugs.openjdk.java.net/browse/JDK-8258981): JVM\n  crash with # Problematic frame: # V [libjvm.so+0xdc0df5]\n  ok_to_convert(Node*, Node*)+0x15\n- [JDK-8271130](https://bugs.openjdk.java.net/browse/JDK-8271130)\n  (Login required): See\n  [CVE-2022-21305](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-21305)\n- [JDK-8271276](https://bugs.openjdk.java.net/browse/JDK-8271276): C2:\n  Wrong JVM state used for receiver null check\n- [JDK-8271459](https://bugs.openjdk.java.net/browse/JDK-8271459): C2:\n  Missing NegativeArraySizeException when creating StringBuilder with\n  negative capacity\n- [JDK-8271926](https://bugs.openjdk.java.net/browse/JDK-8271926):\n  Crash related to Arrays.copyOf with # Problematic frame: # V\n  [libjvm.so+0xc1b83d] NodeHash::hash_delete(Node const*)+0xd\n- [JDK-8297730](https://bugs.openjdk.java.net/browse/JDK-8297730):\n  C2: Arraycopy intrinsic throws incorrect exception\n\n## Citation\n\nIf you use JAttack in your research, we request you to cite our\n[ASE'22 paper](https://cptgit.github.io/dl/papers/zang22jattack.pdf)\n(which won an ACM SIGSOFT Distinguished Paper Award) and [ICSE'23 Demo\npaper](https://cptgit.github.io/dl/papers/zang23jattacktool.pdf).\n\n```bibtex\n@inproceedings{zang22jattack,\n  author = {Zang, Zhiqiang and Wiatrek, Nathaniel and Gligoric, Milos and Shi, August},\n  title = {Compiler Testing using Template {J}ava Programs},\n  booktitle = {International Conference on Automated Software Engineering},\n  pages = {23:1--23:13},\n  year = {2022},\n  doi = {10.1145/3551349.3556958},\n}\n\n@inproceedings{zang23jattacktool,\n  author = {Zang, Zhiqiang and Yu, Fu-Yao and Wiatrek, Nathaniel and Gligoric, Milos and Shi, August},\n  title = {{JA}ttack: {J}ava {JIT} Testing using Template Programs},\n  booktitle = {International Conference on Software Engineering, Tool Demonstrations Track},\n  pages = {6--10},\n  year= {2023},\n  doi = {10.1109/ICSE-Companion58688.2023.00014},\n}\n```\n\n## Contact\n\nLet me ([Zhiqiang Zang](https://github.com/CptGit)) know if you have\nany questions.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fengineeringsoftware%2Fjattack","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fengineeringsoftware%2Fjattack","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fengineeringsoftware%2Fjattack/lists"}