{"id":50677676,"url":"https://github.com/kassane/espressif-toolchains-research","last_synced_at":"2026-06-08T16:35:02.775Z","repository":{"id":362801271,"uuid":"1251596374","full_name":"kassane/espressif-toolchains-research","owner":"kassane","description":"AI Research about espressif/llvm based toolchains support","archived":false,"fork":false,"pushed_at":"2026-06-06T01:40:00.000Z","size":591,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-06T02:12:13.444Z","etag":null,"topics":["compilers","d","espressif","llvm","research","rust","tinygo","toolchain","zig"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"unlicense","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kassane.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-27T18:26:35.000Z","updated_at":"2026-06-06T01:40:03.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/kassane/espressif-toolchains-research","commit_stats":null,"previous_names":["kassane/espressif-toolchains-research"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/kassane/espressif-toolchains-research","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kassane%2Fespressif-toolchains-research","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kassane%2Fespressif-toolchains-research/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kassane%2Fespressif-toolchains-research/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kassane%2Fespressif-toolchains-research/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kassane","download_url":"https://codeload.github.com/kassane/espressif-toolchains-research/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kassane%2Fespressif-toolchains-research/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34071657,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-08T02:00:07.615Z","response_time":111,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["compilers","d","espressif","llvm","research","rust","tinygo","toolchain","zig"],"created_at":"2026-06-08T16:35:02.052Z","updated_at":"2026-06-08T16:35:02.766Z","avatar_url":"https://github.com/kassane.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# espressif-ffi-ai\n\nResearch \u0026 test bed for **cross-language FFI on Espressif's Xtensa (ESP32 / S2\n/ S3) and RISC-V (ESP32-C3)** silicon, riding the shared `espressif/llvm-project`\nbackend. Six toolchains in scope: clang, gcc, rustc, zig, ldc2, tinygo.\n\nThe central question:\n\n\u003e LLVM-frontend toolchains (clang, rustc, zig, ldc2, tinygo) plus a non-LLVM\n\u003e control (gcc) all target Espressif Xtensa through some fork of LLVM. Does\n\u003e that shared backend actually give a shared ABI — can the languages call\n\u003e each other freely on a real ESP32 core, and can their IRs and binaries be\n\u003e mixed?\n\nShort answer, established empirically in this repo:\n\n\u003e **Yes for scalars, floats, pointers, callbacks and struct returns — five\n\u003e co-linkable toolchains (clang, rust, zig, D-LDC, gcc) agree on the ABI\n\u003e (verified in disassembly and live on qemu). TinyGo joins the same backend\n\u003e family on Xtensa but, in v0.41.1, drags its Go runtime into every `.o` so\n\u003e we leave it out of the FFI matrix (docs/24). The holes are in by-value\n\u003e struct *arguments* on the two frontends that defer ABI lowering to the\n\u003e backend: Zig (under-aligned structs on Xtensa; small `{i32,i32}` on RISC-V)\n\u003e and — more broadly — D/LDC (every by-value struct + small-struct return).\n\u003e Rust/clang/gcc are correct everywhere. Fix: pass structs by pointer.**\n\nSee **[Research.md](Research.md)** for the full write-up and **[docs/](docs/)** for\nthe detailed evidence.\n\n## The six toolchains\n\n| Lang | Toolchain | Version | Backend |\n|------|-----------|---------|---------|\n| C/C++ (clang) | [espressif/llvm-project](https://github.com/espressif/llvm-project) `esp-21.1.3_20260408` | clang/LLVM **21.1.3** | LLVM Xtensa |\n| Rust | [esp-rs/rust-build](https://github.com/esp-rs/rust-build) `v1.95.0.0` | rustc 1.95.0-nightly, LLVM **21.1.3** | LLVM Xtensa |\n| Zig | [kassane/zig-espressif-bootstrap](https://github.com/kassane/zig-espressif-bootstrap) `0.17.0-xtensa-dev` (canonical; the `0.16.0-xtensa` tag is the `$ZIG_016` legacy lane) | **Zig 0.17.0-xtensa**, bundled clang/LLVM **22.1.4** | LLVM Xtensa |\n| D | [kassane/esp-idf-dlang](https://github.com/kassane/esp-idf-dlang/releases/tag/xtensa-toolchain) `xtensa-toolchain` (`-betterC`) | **LDC 1.42.0**, espressif/llvm-project **LLVM 22.1.4** (2026-05-30 maintainer re-upload bumped both — docs/05 §\"LDC 1.42 status\", docs/23) | LLVM Xtensa (espressif fork) |\n| Go | [tinygo-org/tinygo](https://github.com/tinygo-org/tinygo/releases/tag/v0.41.1) `v0.41.1` | TinyGo 0.41.1, bundled **LLVM 20.1.1** | LLVM Xtensa (tinygo-org fork; esp32/s3/c3 — no s2) |\n| C/C++ (gcc) | [espressif/crosstool-NG](https://github.com/espressif/crosstool-NG) `esp-15.2.0_20251204` | gcc **15.2.0** | GCC Xtensa (control) |\n\nThe LLVM-frontend toolchains ride a fork of LLVM-Xtensa — clang and rustc on\n`espressif/llvm-project` 21.1.3 (the **LLVM-21 cluster**); **canonical LDC\n1.42.0** on the espressif fork bumped to 22.1.4 (joining the **LLVM-22\ncluster** with zig 0.17 22.1.4 and `$LDC2_UPSTREAM` 22.1.2 — the 2026-05-30\nmaintainer re-upload of `kassane/esp-idf-dlang` bumped LDC AND moved it\nbetween clusters; docs/05 §\"LDC 1.42 status\", docs/23); the legacy\n`$ZIG_016` lane uses bundled 21.1.0; TinyGo on its own bundled\n`tinygo-org/llvm-project` 20.1.1. GCC is the\nnon-LLVM control. TinyGo's output\ndefaults to a full ESP32 flash image but `-o foo.o` does produce a real\nrelocatable Xtensa ELF (with ~196 KB of Go runtime + scheduler undefs — see\ndocs/24 §d for what a consumer must supply). An *optional* `ldc-developers/ldc`\nCI build of LDC on upstream LLVM **22.1.2** (`setup.sh LDC_UPSTREAM=1` →\n`$LDC2_UPSTREAM`) lives only as the \"before\" arm of\n[`experiments/ldc-fork-comparison`](experiments/ldc-fork-comparison/) — see\n[docs/23](docs/23-ldc-espressif-fork.md) for the workarounds the\nespressif-fork LDC removes.\n\n**At-a-glance comparison:** [docs/00-support-matrix.md](docs/00-support-matrix.md)\n(Rust × Zig × D × esp-clang × GCC × TinyGo — versions, targeting, ABI/FFI\ncorrectness, sizes, LTO, mangling). D deep-dive: [docs/19](docs/19-dlang-ldc.md)\n+ [docs/23](docs/23-ldc-espressif-fork.md). TinyGo deep-dive: [docs/24](docs/24-tinygo.md).\n\n## Quick start\n\n```bash\n./scripts/setup.sh          # download + extract + install the 6 toolchains (~1.2 GB)\nsource scripts/env.sh       # point at the toolchains\n./scripts/build-ffi.sh all  # build the FFI matrix: host (runs) + esp32/s2/s3 (link)\n./scripts/analyze.sh esp32  # regenerate IR / disassembly / size evidence\n```\n\nToolchains install **outside** the repo (`/home/user/toolchains`) and are never\ncommitted; `.gitignore` guards against it.\n\n## Layout\n\n```\nexperiments/\n  ffi-matrix/      5 languages implement one C-ABI contract (ffi_abi.h); a C\n                   driver calls all of them. Builds for host + xtensa.\n  abi-structs/     clang/zig/D caller sweep — documents the historical\n                   by-value struct-arg bugs (Zig 0.16 + LDC 1.42-git, both\n                   fixed on canonical; reproduce via `ZIG=$ZIG_016` /\n                   `$LDC2_UPSTREAM`); covers byte arrays, word arrays, AND\n                   C-style bitfields\n  llvm-ir-mix/     cross-language LTO / IR-merge probes (+ LLVM-22 llvm-link merge)\n  dlang/           D/LDC deep-dive: ABI, extern(C++), -HC headers, LTO (docs/19)\n  ldc-fork-comparison/ espressif-21 vs upstream-22 LDC side-by-side (docs/23)\n  atomics-orders/  atomic memory-order parity battery (docs/17 extended)\n  tinygo/          TinyGo v0.41.1 / LLVM 20.1.1 probe; whole-program (docs/24)\n  baremetal-mixin/ runnable use-case: Rust app + Zig kernel in one no_std ELF\n  qemu-run/        bare-metal semihosting harnesses (xtensa + riscv) for qemu\nscripts/           setup / env / build / analyze\ndocs/              detailed findings (00–24: toolchains, ABI, IR, FFI matrix, D safety/features, TMP-FFI, DWARF/codegen audit, LDC espressif-fork, TinyGo)\nResearch.md        headline write-up\nHANDOFF.md         current state + next steps\nCLAUDE.md          orientation for future automated sessions\n```\n\n## Headline results\n\n- **Host (x86_64) FFI matrix runs and passes** — all 45 cross-language calls\n  (C↔C++↔Rust↔Zig↔D, incl. struct-by-value, sret, f32/f64, i64, callbacks).\n  TinyGo is exercised standalone in `experiments/tinygo/` and shares the\n  byte-identical datalayout but stays outside the matrix per docs/24.\n- **All three Xtensa cores link** as one ELF from a mix of compilers, under\n  **both** `ld.lld` **and** GNU `ld`, with **0 unresolved symbols** — including\n  images that mix **GCC-built** and **LLVM-built** (clang/rust/zig/D) objects.\n- **ABI agreement is verifiable in the disassembly**: `entry`/`retw.n` windowed\n  frames, integer args in `a2..a7`, returns in `a2`, callbacks via `callx8` —\n  identical across clang, rust, zig, D, gcc, *and* TinyGo (per its\n  intermediate ELF, docs/22 §g + docs/24).\n- **IR interop**: every LLVM frontend in the matrix shares the byte-identical\n  Xtensa `target datalayout` (clang/rust/zig/D/TinyGo — docs/04). The espressif-fork\n  LDC matches the trio; the upstream-22 LDC used to differ (docs/23); TinyGo\n  on LLVM-20 still matches byte-for-byte (docs/24 §c). Same-version (21.1.3)\n  bitcode is LTO-mergeable across the **LLVM-21 cluster** (clang↔rust↔D);\n  zig 0.17's 22.1.4 sits in a **second LLVM-22 cluster** with the optional\n  upstream LDC + `$LDC_LLVM_DIR` binutils. TinyGo (20.1.1) is outside both.\n  The LLVM-22 `llvm-link` reads esp-clang 21.1.3 bitcode fine, so cross-\n  cluster IR merging works (docs/04).\n- **One frontend mis-handles by-value struct *arguments*** on the canonical\n  lane (Rust/clang/gcc/Zig 0.17/LDC 1.42.0 all correct): **TinyGo** lowers\n  `struct{[N]uint8}` as `[N x i8]` byte-per-register (docs/24 §e), so byte-\n  array aggregates round-trip incorrectly on Xtensa. Two historical outliers\n  (Zig 0.16 align-1 + small `{i32,i32}`, and the previous LDC 1.42-git's\n  universal `byval/sret` lowering) are closed on the canonical `$ZIG` /\n  `$LDC2` lanes; reproducers preserved on `$ZIG_016` / `$LDC2_UPSTREAM`.\n  Full bug-fix narrative + IR shapes in docs/05 §\"Zig 0.17 status\" +\n  §\"LDC 1.42 status\". Struct returns ≤reg-size, scalars, pointers, and\n  callbacks are fine everywhere; **pass structs by pointer** across a Zig,\n  D, or TinyGo-byte-array boundary.\n- **Confirmed at runtime on qemu** (both `qemu-system-xtensa` and\n  `qemu-system-riscv32`): on the **canonical lane** (`$ZIG` 0.17 + `$LDC2`\n  1.42.0), every FFI-matrix language passes — xtensa qemu reports 0 failures.\n  On the legacy lanes (`$ZIG_016` / `$LDC2_UPSTREAM`) the historical breaks\n  reproduce: xtensa `zig blob_sum FAIL` + `d point_dot/d blob_sum FAIL`;\n  riscv `zig point_dot FAIL`. TinyGo is out-of-matrix.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkassane%2Fespressif-toolchains-research","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkassane%2Fespressif-toolchains-research","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkassane%2Fespressif-toolchains-research/lists"}