{"id":13730025,"url":"https://github.com/HexHive/retrowrite","last_synced_at":"2025-05-08T02:31:12.332Z","repository":{"id":37732936,"uuid":"146674758","full_name":"HexHive/retrowrite","owner":"HexHive","description":"RetroWrite --  Retrofitting compiler passes through binary rewriting","archived":false,"fork":false,"pushed_at":"2024-05-13T08:56:59.000Z","size":3679,"stargazers_count":674,"open_issues_count":32,"forks_count":77,"subscribers_count":37,"default_branch":"master","last_synced_at":"2024-11-05T04:51:32.731Z","etag":null,"topics":["aarch64","assembly","binary-rewriting","disassembler","reverse-engineering","security","x86-64"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HexHive.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2018-08-30T00:35:56.000Z","updated_at":"2024-11-02T16:19:26.000Z","dependencies_parsed_at":"2023-11-28T02:26:10.009Z","dependency_job_id":"ee98808a-14a2-405d-a2b6-2304a0a06096","html_url":"https://github.com/HexHive/retrowrite","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HexHive%2Fretrowrite","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HexHive%2Fretrowrite/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HexHive%2Fretrowrite/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HexHive%2Fretrowrite/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HexHive","download_url":"https://codeload.github.com/HexHive/retrowrite/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224689278,"owners_count":17353354,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aarch64","assembly","binary-rewriting","disassembler","reverse-engineering","security","x86-64"],"created_at":"2024-08-03T02:01:08.884Z","updated_at":"2025-05-08T02:31:12.322Z","avatar_url":"https://github.com/HexHive.png","language":"Python","readme":"# Retrowrite\n\nRetrowrite is a static binary rewriter for x64 and aarch64. It works without\nheuristics, does not introduce overhead and uses the *symbolization* technique\n(also known as *reassemblable assembly*) to insert instrumentation to binaries\nwithout the need for source code.\n\nPlease note that the x64 version and the arm64 version use different rewriting algorithms and\nsupport a different set of features. \n\nFor technical details, you can read the \n[paper](https://nebelwelt.net/publications/files/20Oakland.pdf)\n(in *IEEE S\u0026P'20*) for the x64 version and this [thesis](https://hexhive.epfl.ch/theses/20-dibartolomeo-thesis.pdf) \nfor the arm64 version. \n\n[KRetrowrite](#kretrowrite) is a variant of the x64 version that supports the rewriting\nof Linux kernel modules. \n\n\u003c!--Code repository for \"Retrowrite: Statically Instrumenting COTS Binaries for--\u003e\n\u003c!--Fuzzing and Sanitization\" (in *IEEE S\u0026P'20*). Please refer to the--\u003e\n\u003c!--[paper](https://nebelwelt.net/publications/files/20Oakland.pdf) for--\u003e\n\u003c!--technical details. There's also a--\u003e\n\u003c!--[36c3 presentation](http://nebelwelt.net/publications/files/19CCC-presentation.pdf)--\u003e\n\u003c!--and [36c3 video](https://media.ccc.de/v/36c3-10880-no_source_no_problem_high_speed_binary_fuzzing)--\u003e\n\u003c!--to get you started.--\u003e\n\n\u003c!--This project contains 2 different version of retrowrite :--\u003e\n\u003c!--* [Retrowrite](#retrowrite-1) to rewrite classic userspace binaries, and--\u003e\n\u003c!--* [KRetrowrite](#kretrowrite) to rewrite and fuzz kernel modules.--\u003e\n\n\u003c!--The two versions can be used independently of each other or at the same time.--\u003e\n\u003c!--In case you want to use both please follow the instructions for KRetrowrite.--\u003e\n\n## General setup\n\nRetrowrite is implemented in python3 (3.6). It depends on `pyelftools` and `capstone`.\nTo install the dependencies, please run:\n```python\npip install -r requirements.txt\n```\nIt is not recommended to install the dependencies from your distro's package managers, as they\nmight be outdated.\n\n\n\u003c!--Make sure python3 and python3-venv--\u003e\n\u003c!--is installed on system. Retrowrite depends on--\u003e\n\u003c!--[capstone](https://github.com/aquynh/capstone). The version--\u003e\n\u003c!--available from the Ubuntu 18.04 repositories --\u003e\n\u003c!--is not compatible with this version. The setup--\u003e\n\u003c!--script pulls the latest version of capstone from the repository and builds it.--\u003e\n\u003c!--Make sure that your system meets the requirements to build capstone.--\u003e\n\n#### Features\n\n|                              | retrowrite-x64     | retrowrite-aarch64 |\n|------------------------------|--------------------|--------------------|\n| stripped binaries            | :x: (WIP)          | :white_check_mark: |\n| Non-PIE binaries             | :x:                | :white_check_mark: |\n| Non-standard compilers       | :x:                | :white_check_mark: |\n| Zero overhead                | :white_check_mark: | :white_check_mark: |\n| Kernel modules support       | :white_check_mark: | :x:                |\n| AFL-coverage instrumentation | :white_check_mark: | :white_check_mark: |\n| ASan instrumentation         | :white_check_mark: | :white_check_mark: |\n| C++ support                  | :x: (WIP)          | :x: (WIP)          |\n\n\n#### Command line options\n\n```bash\n(retro) $ retrowrite --help\nusage: retrowrite [-h] [-a] [-A] [-m MODULE] [-k] [--kcov] [-c] [--ignore-no-pie] [--ignore-stripped] [-v] bin outfile\n\npositional arguments:\n  bin                   Input binary to load\n  outfile               Symbolized ASM output\n\noptional arguments:\n  -h, --help            show this help message and exit\n  -a, --assemble        Assemble instrumented assembly file into instrumented binary\n  -A, --asan            Add binary address sanitizer instrumentation\n  -m MODULE, --module MODULE\n                        Use specified instrumentation pass/module in rwtools directory\n  -k, --kernel          Instrument a kernel module\n  --kcov                Instrument the kernel module with kcov\n  -c, --cache           Save/load register analysis cache (only used with --asan)\n  --ignore-no-pie       Ignore position-independent-executable check (use with caution)\n  --ignore-stripped     Ignore stripped executable check (use with caution)\n  -v, --verbose         Verbose output\n```\n\n### Instrumentation passes\n\nSelect the instrumentation pass you would like to apply with `retrowrite -m \u003cpass\u003e`\nYou can find the available instrumentation passes in folders `rwtools_x64` and `rwtools_arm64`.\n\nAvailable instrumentation passes for x64:\n\t- AddressSanitizer\n\t- AFL-coverage information\n\nAvailable instrumentation passes for aarch64:\n\t- AddressSanitizer\n\t- AFL-coverage information + forkserver\n\t- Coarse grained control flow integrity on function entries\n\n\n\n\n\u003c!--In case you load a non position independent code you will get the following message:--\u003e\n\u003c!--```--\u003e\n\u003c!--(retro) $ retrowrite stack stack.c --\u003e\n\u003c!--***** RetroWrite requires a position-independent executable. *****--\u003e\n\u003c!--It looks like stack is not position independent--\u003e\n\u003c!--If you really want to continue, because you think retrowrite has made a mistake, pass --ignore-no-pie.--\u003e\n\u003c!--```--\u003e\n\u003c!--In the case you think retrowrite is mistaking you can use the argument `--ignore-no-pie`.--\u003e\n\n\n\u003c!--## Retrowrite--\u003e\n\u003c!--### Quick Usage Guide--\u003e\n\n\u003c!--This section highlights the steps to get you up to speed to use userspace retrowrite for rewriting PIC binaries.--\u003e\n\n\u003c!--Retrowrite ships with an utility with the following features:--\u003e\n\u003c!--* Generate symbolized assembly files from binaries without source code--\u003e\n\u003c!--* BASan: Instrument binary with binary-only Address Sanitizer --\u003e\n\u003c!--* Support for symbolizing (linux) kernel modules --\u003e\n\u003c!--* KCovariance instrumentation support--\u003e\n\n\u003c!--### Usage--\u003e\n\n\u003c!--#### Commands--\u003e\n\n\n\n## Example usage\n\n#### a. Instrument Binary with Binary-Address Sanitizer (BASan)\n\n`retrowrite --asan \u003c/path/to/binary/\u003e \u003c/path/to/output/binary\u003e`\n\nNote: If on x64, make sure that the binary is position-independent and is not stripped.\nThis can be checked using `file` command (the output should say `ELF shared object`).\n\nExample, create an instrumented version of `/bin/ls`:\n\n`retrowrite --asan /bin/ls ls-basan-instrumented.s`\n\nThis will generate an assembly (`.s`) file. \nTo recompile the assembly back into a binary, it depends on the architecture:\n\n##### x64\nThe generated assembly can be assembled and linked\nusing any compiler, like:\n\n`gcc ls-basan-instrumented.s -lasan -o ls-basan-instrumented`\n\n**debug** in case you get the error ```undefined reference to `__asan_init_v4'``` , \nreplace \"asan_init_v4\" by \"asan_init\"  in the assembly file, the following command can help you do that:\n```sed -i 's/asan_init_v4/asan_init/g' ls-basan-instrumented.s```\n\n##### aarch64\nOn aarch64, we also rely on standard compilers to assemble and link but the collection of compiler \nflags is slightly more involved and so we provide the `-a` switch on the main `retrowrite` \nexecutable to do that for you:\n\n`retrowrite -a ls-basan-instrumented.s -lasan -o ls-basan-instrumented`\n\n\n#### b. Instrument a binary with coverage information and fuzz with AFL\n\n##### x64\n\nTo generate an AFL-instrumented binary, first generate the symbolized assembly\nas described above. Then, recompile the symbolized assembly with `afl-gcc` from\n[afl++](https://github.com/vanhauser-thc/AFLplusplus) like this:\n\n```\n$ AFL_AS_FORCE_INSTRUMENT=1 afl-gcc foo.s -o foo\n```\n or `afl-clang`.\n\n##### aarch64\n\nTo instrument a binary with coverage information, use the coverage instrumentation pass\nwith `retrowrite -m coverage \u003cinput file\u003e \u003coutput asm\u003e`.  Re-assemble the binary \nwith `retrowrite -a \u003coutput asm\u003e \u003cnew binary\u003e`.\n\nThe binary can now be fuzzed with:\n```bash\nafl-fuzz -i \u003cseed folder\u003e -o \u003cout folder\u003e \u003cnew binary\u003e\n```\n\nRetrowrite also tries to add instrumentation to act as a forkserver for AFL; in case this \ncauses problems, you can disable this behaviour by using `export AFL_NO_FORKSERVER=1`\n\n#### c. Generate Symbolized Assembly\n\nTo generate symbolized assembly that may be modified by hand or post-processed\nby existing tools, just do not specify any instrumentation pass:\n\n`retrowrite \u003c/path/to/binary\u003e \u003cpath/to/output/asm/files\u003e`\n\nThe output asm files can be freely edited by hand or by other tools.\nPost-modification, the asm files may be assembled to working binaries as\ndescribed above.\n\nWhile retrowrite is interoperable with other tools, we\nstrongly encourage researchers to use the retrowrite API for their binary\ninstrumentation / modification needs! This saves the additional effort of\nhaving to load and parse binaries or assembly files. \n\n\u003c!--## Docker / Reproducing Results--\u003e\n\n\u003c!--See [fuzzing/docker](fuzzing/docker) for more information on building a docker image for--\u003e\n\u003c!--fuzzing and reproducing results.--\u003e\n\n#### d. Obfuscate a binary by randomizing the order of instructions (x86 only for now):\n\nThe `jumparound` pass randomizes the order of instructions in a function. It can be found\nin folder `rwtools_x64/jumparound`. To use it, you need to specify the `-m` option as following:\n\n`retrowrite -m jumparound \u003cinput file\u003e \u003coutput asm\u003e`\n\n\n\n# KRetrowrite\n### Quick Usage Guide\n### Setup\n\nRun `setup.sh`:\n\n* `./setup.sh kernel`\n\n\nActivate the virtualenv (from root of the repository):\n\n* `source retro/bin/activate`\n\n(Bonus) To exit virtualenv when you're done with retrowrite:\n* `deactivate`\n\n\n### Usage\n\n\n#### Commands\n\n##### Classic instrumentation\n\n* Instrument Binary with Binary-Address Sanitizer (BASan)  :`retrowrite --asan --kernel \u003c/path/to/module.ko\u003e \u003c/path/to/output/module_asan.ko\u003e`\n* Generate Symbolized Assembly that may be modified by hand or post-processed by existing tools: `retrowrite \u003c/path/to/module.ko\u003e \u003cpath/to/output/asm/files\u003e`\n\n##### Fuzzing\n\nFor fuzzing campaign please see [fuzzing/](fuzzing/) folder.\n\n# Developer Guide\n\nIn general, `librw/` contains the code for loading, disassembly, and\nsymbolization of binaries and forms the core of all transformations.\nIndividual transformation passes that build on top this rewriting framework,\nsuch as our binary-only Address Sanitizer (BASan) is contained as individual\ntools in `rwtools/`.\n\nThe files and folder starting with `k` are linked with the kernel retrowrite version.\n\n# Demos\n\nIn the [demos/](demos/) folder, you will find examples for userspace and kernel retrowrite\n([demos/user_demo](demos/user_demo) and [demos/kernel_demo](demos/kernel_demo) respectively).\n\n\n## Cite\n\nThe following publications cover different parts of the RetroWrite project:\n\n* *RetroWrite: Statically Instrumenting COTS Binaries for Fuzzing and Sanitization*\n  Sushant Dinesh, Nathan Burow, Dongyan Xu, and Mathias Payer.\n  **In Oakland'20: IEEE International Symposium on Security and Privacy, 2020**\n\n* *No source, no problem! High speed binary fuzzing*\n  Matteo Rizzo, and Mathias Payer.\n  **In 36c3'19: Chaos Communication Congress, 2019**\n\n\n# License -- MIT\n\nThe MIT License\n\nCopyright (c) 2019 HexHive Group,\nSushant Dinesh \u003csushant.dinesh94@gmail.com\u003e,\nLuca Di Bartolomeo \u003clucadb96@gmail.com\u003e,\nAntony Vennard \u003cantony@vennard.ch\u003e,\nMatteo Rizzo \u003cmatteorizzo.personal@gmail.com\u003e,\nMathias Payer \u003cmathias.payer@nebelwelt.net\u003e\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in\nall copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN\nTHE SOFTWARE.\n","funding_links":[],"categories":["Code-gen","Resources","C/C++"],"sub_categories":["By Purpose","Address Sanity"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FHexHive%2Fretrowrite","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FHexHive%2Fretrowrite","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FHexHive%2Fretrowrite/lists"}