{"id":13691795,"url":"https://github.com/encounter/decomp-toolkit","last_synced_at":"2025-04-05T01:04:16.397Z","repository":{"id":63877677,"uuid":"571334434","full_name":"encounter/decomp-toolkit","owner":"encounter","description":"A GameCube \u0026 Wii decompilation toolkit","archived":false,"fork":false,"pushed_at":"2025-03-21T22:14:48.000Z","size":1407,"stargazers_count":134,"open_issues_count":19,"forks_count":19,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-29T00:05:06.925Z","etag":null,"topics":["codewarrior","decompilation","disassembler","dol","gamecube","metrowerks","powerpc","rel","reverse-engineering","rust","wii"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/encounter.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-11-27T22:03:11.000Z","updated_at":"2025-03-21T22:14:52.000Z","dependencies_parsed_at":"2023-12-21T06:30:59.963Z","dependency_job_id":"115c2683-6f66-40c3-ba83-722522aa0701","html_url":"https://github.com/encounter/decomp-toolkit","commit_stats":null,"previous_names":[],"tags_count":60,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/encounter%2Fdecomp-toolkit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/encounter%2Fdecomp-toolkit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/encounter%2Fdecomp-toolkit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/encounter%2Fdecomp-toolkit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/encounter","download_url":"https://codeload.github.com/encounter/decomp-toolkit/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247271517,"owners_count":20911587,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["codewarrior","decompilation","disassembler","dol","gamecube","metrowerks","powerpc","rel","reverse-engineering","rust","wii"],"created_at":"2024-08-02T17:00:50.506Z","updated_at":"2025-04-05T01:04:16.378Z","avatar_url":"https://github.com/encounter.png","language":"Rust","funding_links":[],"categories":["Tools and Libraries","🛠️ General Tools"],"sub_categories":["🔬 Format Analysis \u0026 Reverse Engineering"],"readme":"# decomp-toolkit [![Build Status]][actions]\n\n[Build Status]: https://github.com/encounter/decomp-toolkit/actions/workflows/build.yml/badge.svg\n[actions]: https://github.com/encounter/decomp-toolkit/actions\n\nYet another GameCube/Wii decompilation toolkit.\n\ndecomp-toolkit functions both as a command-line tool for developers, and as a replacement for various parts of a\ndecompilation project's build system.\n\nFor use in a new decompilation project, see [dtk-template](https://github.com/encounter/dtk-template), which provides a\nproject structure and build system that uses decomp-toolkit under the hood.\n\n## Sections\n\n- [Goals](#goals)\n- [Background](#background)\n- [Analyzer features](#analyzer-features)\n- [Other approaches](docs/other_approaches.md)\n- [Terminology](docs/terminology.md)\n- [Commands](#commands)\n  - [ar create](#ar-create)\n  - [ar extract](#ar-extract)\n  - [demangle](#demangle)\n  - [disc info](#disc-info)\n  - [disc extract](#disc-extract)\n  - [disc convert](#disc-convert)\n  - [disc verify](#disc-verify)\n  - [dol info](#dol-info)\n  - [dol split](#dol-split)\n  - [dol diff](#dol-diff)\n  - [dol apply](#dol-apply)\n  - [dol config](#dol-config)\n  - [dwarf dump](#dwarf-dump)\n  - [elf disasm](#elf-disasm)\n  - [elf fixup](#elf-fixup)\n  - [elf2dol](#elf2dol)\n  - [map](#map)\n  - [rel info](#rel-info)\n  - [rel merge](#rel-merge)\n  - [rso info](#rso-info)\n  - [rso make](#rso-make)\n  - [shasum](#shasum)\n  - [nlzss decompress](#nlzss-decompress)\n  - [rarc list](#rarc-list)\n  - [rarc extract](#rarc-extract)\n  - [u8 list](#u8-list)\n  - [u8 extract](#u8-extract)\n  - [vfs ls](#vfs-ls)\n  - [vfs cp](#vfs-cp)\n  - [yay0 decompress](#yay0-decompress)\n  - [yay0 compress](#yay0-compress)\n  - [yaz0 decompress](#yaz0-decompress)\n  - [yaz0 compress](#yaz0-compress)\n  - [wad info](#wad-info)\n  - [wad extract](#wad-extract)\n  - [wad verify](#wad-verify)\n\n## Goals\n\n- Automate as much as possible, allowing developers to focus on matching code rather than months-long tedious setup.\n- Provide highly **accurate** and performant analysis and tooling.\n- Provide everything in a single portable binary. This simplifies project setup: a script can simply fetch the\n  binary from GitHub.\n- Replace common usages of msys2 and GNU assembler, eliminating the need to depend on devkitPro.\n- Integrate well with other decompilation tooling like [objdiff](https://github.com/encounter/objdiff) and\n  [decomp.me](https://decomp.me).\n\n## Background\n\nThe goal of a matching decompilation project is to write C/C++ code that compiles back to the _exact_ same binary as\nthe original game. This often requires using the same compiler as the original game. (For GameCube and Wii,\n[Metrowerks CodeWarrior](https://en.wikipedia.org/wiki/CodeWarrior))\n\nWhen compiling C/C++ code, the compiler (in our case, `mwcceppc`) generates an object file (`.o`) for every source file.\nThis object file contains the compiled machine code, as well as information that the linker (`mwldeppc`) uses to\ngenerate the final executable.\n\nOne way to verify that our code is a match is by taking any code that has been decompiled, and\nlinking it alongside portions of the original binary that have not been decompiled yet. First, we create relocatable\nobjects from the original binary:\n\n\u003cpicture\u003e\n  \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"assets/diagram_dark.svg\"\u003e\n  \u003csource media=\"(prefers-color-scheme: light)\" srcset=\"assets/diagram_light.svg\"\u003e\n  \u003cimg alt=\"Binary split diagram\" src=\"assets/diagram.svg\"\u003e\n\u003c/picture\u003e\n\n(Heavily simplified)\n\nThen, each object can be replaced by a decompiled version as matching code is written. If the linker still generates a\nbinary that is byte-for-byte identical to the original, then we know that the decompiled code is a match.\n\ndecomp-toolkit provides tooling for analyzing and splitting the original binary into relocatable objects, as well\nas generating the linker script and other files needed to link the decompiled code.\n\n## Analyzer features\n\n**Function boundary analysis**  \nDiscovers function boundaries with high accuracy. Uses various heuristics to disambiguate tail calls from\ninner-function control flow.\n\n**Signature analysis**  \nUtilizes a built-in signature database to identify common Metrowerks and SDK functions and objects.  \nThis also helps decomp-toolkit automatically generate required splits, like `__init_cpp_exceptions`.\n\n**Relocation analysis**  \nPerforms control-flow analysis and rebuilds relocations with high accuracy.  \nWith some manual tweaking (mainly in data), this should generate fully-shiftable objects.\n\n**Section analysis**  \nAutomatically identifies DOL and REL sections based on information from signature and relocation analysis.\n\n**Object analysis**  \nAttempts to identify the type and size of data objects by analyzing usage.  \nAlso attempts to identify string literals, wide string literals, and string tables.\n\n**Splitting**  \nGenerates split object files in memory based on user configuration.  \nIn order to support relinking with `mwldeppc.exe`, any **unsplit** `.ctors`, `.dtors`, `extab` and `extabindex` entries\nare analyzed and automatically split along with their associated functions. This ensures that the linker will properly\ngenerate these sections without any additional configuration.  \nA topological sort is performed to determine the final link order of the split objects.\n\n**Object file writing**  \nWrites object files directly, with no assembler required. (Bye devkitPPC!)  \nIf desired, optionally writes GNU assembler-compatible files alongside the object files.\n\n**Linker script generation**  \nGenerates `ldscript.lcf` for `mwldeppc.exe`.\n\n## Commands\n\n### ar create\n\nCreate a static library (.a) from the input objects.\n\n```shell\n$ dtk ar create out.a input_1.o input_2.o\n# or\n$ echo input_1.o \u003e\u003e rspfile\n$ echo input_2.o \u003e\u003e rspfile\n$ dtk ar create out.a @rspfile\n```\n\n### ar extract\n\nExtracts the contents of static library (.a) files.\n\nAccepts multiple files, glob patterns (e.g. `*.a`) and response files (e.g. `@rspfile`).\n\nOptions:\n- `-o`, `--out \u003coutput-dir\u003e`: Output directory. Defaults to the current directory.\n- `-v`, `--verbose`: Verbose output.\n- `-q`, `--quiet`: Suppresses all output except errors.\n\n```shell\n# Extracts to outdir\n$ dtk ar extract lib.a -o outdir\n\n# With multiple inputs, extracts to separate directories\n# Extracts to outdir/lib1, outdir/lib2\n$ dtk ar extract lib1.a lib2.a -o outdir\n```\n\n### demangle\n\nDemangles CodeWarrior C++ symbols. A thin wrapper for [cwdemangle](https://github.com/encounter/cwdemangle).\n\n```shell\n$ dtk demangle 'BuildLight__9CGuiLightCFv'\nCGuiLight::BuildLight() const\n```\n\n### disc info\n\n_`disc` commands are wrappers around the [nod](https://github.com/encounter/nod-rs) library\nand its `nodtool` command line tool._\n\nDisplays information about disc images.\n\nTo list the contents of a disc image, use [vfs ls](#vfs-ls).\n\nSupported disc image formats:\n\n- ISO (GCM)\n- WIA / RVZ\n- WBFS (+ NKit 2 lossless)\n- CISO (+ NKit 2 lossless)\n- NFS (Wii U VC)\n- GCZ\n- TGC\n\n```shell\n$ dtk disc info /path/to/game.iso\n```\n\n### disc extract\n\nExtracts the contents of disc images to a directory.\n\nSee [disc info](#disc-info) for supported formats.\n\n\u003e [!NOTE]  \n\u003e [vfs cp](#vfs-cp) is more flexible and supports disc images.\n\n```shell\n$ dtk disc extract /path/to/game.iso [outdir]\n```\n\nBy default, only the main **data** partition is extracted.  \nUse the `-p`/`--partition` option to choose a different partition.  \n(Options: `all`, `data`, `update`, `channel`, or a partition index)\n\n### disc convert\n\nConverts any supported disc image to raw ISO (GCM).\n\nIf the format is lossless, the output will be identical to the original disc image.\n\nSee [disc info](#disc-info) for supported formats.\n\n```shell\n$ dtk disc convert /path/to/game.wia /path/to/game.iso\n```\n\n### disc verify\n\nHashes the contents of a disc image and verifies it against a built-in [Redump](http://redump.org/) database.\n\nSee [disc info](#disc-info) for supported formats.\n\n```shell\n$ dtk disc verify /path/to/game.iso\n```\n\n### dol info\n\nAnalyzes a DOL file and outputs information section and symbol information.\n\nSee [vfs ls](#vfs-ls) for information on the VFS abstraction.\n\n```shell\n$ dtk dol info input.dol\n# or, directly from a disc image\n$ dtk dol info 'disc.rvz:sys/main.dol'\n```\n\n### dol split\n\n\u003e [!IMPORTANT]  \n\u003e **This command is intended to be used as part of a decompilation project's build system.**  \n\u003e For an example project structure and for documentation on the configuration, see\n\u003e [dtk-template](https://github.com/encounter/dtk-template).\n\nAnalyzes and splits a DOL file into relocatable objects based on user configuration.\n\n```shell\n$ dtk dol split config.yml target\n```\n\n### dol diff\n\nSimple diff tool for issues in a linked ELF. (Yes, not DOL. It's misnamed.)  \nTries to find the most obvious difference causing a mismatch.\n\nPass in the project configuration file, and the path to the linked ELF file to compare against.\n\n```shell\n$ dtk dol diff config.yml build/main.elf\n```\n\n### dol apply\n\nApplies updated symbols from a linked ELF to the project configuration. (Again, misnamed.)\n\nUseful after matching a file. It will pull updated symbol information from the final result.\n\n```shell\n$ dtk dol apply config.yml build/main.elf\n```\n\n### dol config\n\nGenerates an initial project configuration file from a DOL (\u0026 RELs).\n\nPass in the DOL file, and any REL files that are linked with it.  \nOr, for Wii games, pass in the `selfile.sel`. (Not RSOs)\n\n```shell\n$ dtk dol config main.dol rels/*.rel -o config.yml\n```\n\n### dwarf dump\n\nDumps DWARF 1.1 information from an ELF file. (Does **not** support DWARF 2+)\n\n```shell\n$ dtk dwarf dump input.elf\n```\n\n### elf disasm\n\nDisassemble an unstripped CodeWarrior ELF file. Attempts to automatically split objects and rebuild relocations\nwhen possible.\n\n```shell\n$ dtk elf disasm input.elf out\n```\n\n### elf fixup\n\nFixes issues with GNU assembler-built objects to ensure compatibility with `mwldeppc.exe`.\n\n- Strips empty sections\n- Generates section symbols for all allocatable sections\n- Where possible, replaces section-relative relocations with direct relocations.\n- Adds an ` (asm)` suffix to the file symbol. (For matching progress calculation)\n\n```shell\n# input and output can be the same\n$ dtk elf fixup file.o file.o\n```\n\n### elf2dol\n\nCreates a DOL file from the provided ELF file.\n\n```shell\n$ dtk elf2dol input.elf output.dol\n# or, to ignore certain sections\n$ dtk elf2dol input.elf output.dol --ignore debug_section1 --ignore debug_section2\n```\n\n### map\n\nProcesses CodeWarrior map files and provides information about symbols and TUs.\n\n```shell\n$ dtk map entries Game.MAP 'Unit.o'\n# Outputs all symbols that are referenced by Unit.o\n# This is useful for finding deduplicated weak functions,\n# which only show on first use in the link map.\n\n$ dtk map symbol Game.MAP 'Function__5ClassFv'\n# Outputs reference information for Function__5ClassFv\n# CodeWarrior link maps can get very deeply nested,\n# so this is useful for emitting direct references\n# in a readable format.\n```\n\n### rel info\n\nPrints information about a REL file.\n\nSee [vfs ls](#vfs-ls) for information on the VFS abstraction.\n\n```shell\n$ dtk rel info input.rel\n# or, directly from a disc image\n$ dtk rel info 'disc.rvz:files/RELS.arc:amem/d_a_tag_so.rel'\n```\n\n### rel merge\n\nMerges a DOL file and associated RELs into a single ELF file, suitable for analysis in your favorite\nreverse engineering software.\n\n```shell\n$ dtk rel info main.dol rels/*.rel -o merged.elf\n```\n\n### rso info\n\n\u003e [!WARNING]  \n\u003e This command is not yet functional.\n\nPrints information about an RSO file.\n\n```shell\n$ dtk rso info input.rso\n```\n\n### rso make\n\n\u003e [!WARNING]  \n\u003e This command does not yet support creating SEL files.\n\nCreates an RSO file from a relocatable ELF file.\n\nOptions:\n- `-o`, `--output \u003cFile\u003e`: Output RSO file.\n- `-m`, `--module-name \u003cName\u003e`: Module name (or path). Default: input name\n- `-e`, `--export \u003cFile\u003e`: File containing exported symbol names. (Newline separated)\n\n```shell\n$ dtk rso make input.elf -o input.rso\n```\n\n### shasum\n\nCalculate and verify SHA-1 hashes.\n\n```shell\n$ dtk shasum baserom.dol\n949c5ed7368aef547e0b0db1c3678f466e2afbff  baserom.dol\n\n$ dtk shasum -c baserom.sha1 \nbaserom.dol: OK\n```\n\n### nlzss decompress\n\nDecompresses NLZSS-compressed files.\n\n```shell\n$ dtk nlzss decompress input.bin.lz -o output.bin\n# or, for batch processing\n$ dtk nlzss decompress rels/*.lz -o rels\n```\n\n### rarc list\n\n\u003e [!NOTE]  \n\u003e [vfs ls](#vfs-ls) is more flexible and supports RARC archives.  \n\u003e This command is now equivalent to `dtk vfs ls -r input.arc:`\n\nLists the contents of an RARC (older .arc) archive.\n\n```shell\n$ dtk rarc list input.arc\n```\n\n### rarc extract\n\n\u003e [!NOTE]  \n\u003e [vfs cp](#vfs-cp) is more flexible and supports RARC archives.  \n\u003e This command is now equivalent to `dtk vfs cp input.arc: output_dir`\n\nExtracts the contents of an RARC (older .arc) archive.\n\n```shell\n$ dtk rarc extract input.arc -o output_dir\n```\n\n### u8 list\n\n\u003e [!NOTE]\n\u003e [vfs ls](#vfs-ls) is more flexible and supports U8 archives.  \n\u003e This command is now equivalent to `dtk vfs ls -r input.arc:`\n\nExtracts the contents of a U8 (newer .arc) archive.\n\n```shell\n$ dtk u8 list input.arc\n```\n\n### u8 extract\n\n\u003e [!NOTE]\n\u003e [vfs cp](#vfs-cp) is more flexible and supports U8 archives.  \n\u003e This command is now equivalent to `dtk vfs cp input.arc: output_dir`\n\nExtracts the contents of a U8 (newer .arc) archive.\n\n```shell\n$ dtk u8 extract input.arc -o output_dir\n```\n\n### vfs ls\n\ndecomp-toolkit has a powerful virtual filesystem (VFS) abstraction that allows you to work with a\nvariety of containers. All operations happen in memory with minimal overhead and no temporary files.\n\nSupported containers:\n\n- Disc images (see [disc info](#disc-info) for supported formats)\n- RARC archives (older .arc)\n- U8 archives (newer .arc)\n- WAD files (Wii VC)\n\nSupported compression formats are handled transparently:\n- Yay0 (SZP) / Yaz0 (SZS)\n- NLZSS (.lz) (Use `:nlzss` in the path)\n\n`vfs ls` lists the contents of a container or directory.\n\nOptions:\n\n- `-r`, `--recursive`: Recursively list contents.\n- `-s`, `--short`: Only list file names.\n\nExamples:\n\n```shell\n# List the contents of the `amem` directory inside `RELS.arc` in a disc image\n$ dtk vfs ls 'disc.rvz:files/RELS.arc:amem'\n# List the contents of `RELS.arc` recursively\n$ dtk vfs ls -r 'disc.rvz:files/RELS.arc:'\n\n# All commands that accept a file path can also accept a VFS path\n$ dtk rel info 'disc.rvz:files/RELS.arc:amem/d_a_tag_so.rel'\n# Example disc image within a disc image\n$ dtk dol info 'disc.rvz:files/zz_demo.tgc:sys/main.dol'\n````\n\n### vfs cp\n\nSee [vfs ls](#vfs-ls) for information on the VFS abstraction.\n\n`vfs cp` copies files and directories recursively to the host filesystem.\n\nOptions:\n\n- `--no-decompress`: Do not decompress files when copying.\n- `-q`, `--quiet`: Suppresses all output except errors.\n\nExamples:\n\n```shell\n# Extract a file from a nested path in a disc image to the current directory\n$ dtk vfs cp 'disc.rvz:files/RELS.arc:amem/d_a_tag_so.rel' .\n\n# Directories are copied recursively, making it easy to extract entire archives\n$ dtk vfs cp 'disc.rvz:files/RELS.arc:' rels\n# Or, to disable automatic decompression\n$ dtk vfs cp --no-decompress 'disc.rvz:files/RELS.arc:' rels\n```\n\n### yay0 decompress\n\nDecompresses Yay0-compressed files.\n\n```shell\n$ dtk yay0 decompress input.bin.yay0 -o output.bin\n# or, for batch processing\n$ dtk yay0 decompress rels/*.yay0 -o rels\n```\n\n### yay0 compress\n\nCompresses files using Yay0 compression.\n\n```shell\n$ dtk yay0 compress input.bin -o output.bin.yay0\n# or, for batch processing\n$ dtk yay0 compress rels/* -o rels\n```\n\n### yaz0 decompress\n\nDecompresses Yaz0-compressed files.\n\n```shell\n$ dtk yaz0 decompress input.bin.yaz0 -o output.bin\n# or, for batch processing\n$ dtk yaz0 decompress rels/*.yaz0 -o rels\n```\n\n### yaz0 compress\n\nCompresses files using Yaz0 compression.\n\n```shell\n$ dtk yaz0 compress input.bin -o output.bin.yaz0\n# or, for batch processing\n$ dtk yaz0 compress rels/* -o rels\n```\n\n### wad info\n\nPrints information about a WAD file.\n\n```shell\n$ dtk wad info input.wad\n```\n\n### wad extract\n\n\u003e [!NOTE]\n\u003e [vfs cp](#vfs-cp) is more flexible and supports WAD files.  \n\u003e This command is now equivalent to `dtk vfs cp input.wad: output_dir`\n\nExtracts the contents of a WAD file.\n\n```shell\n$ dtk wad extract input.wad -o output_dir\n```\n\n### wad verify\n\nVerifies the contents of a WAD file.\n\n```shell\n$ dtk wad verify input.wad\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fencounter%2Fdecomp-toolkit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fencounter%2Fdecomp-toolkit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fencounter%2Fdecomp-toolkit/lists"}