{"id":32155218,"url":"https://github.com/obsidianjulua/replibuild.jl","last_synced_at":"2026-04-11T10:08:25.846Z","repository":{"id":318735902,"uuid":"1073964059","full_name":"obsidianjulua/RepliBuild.jl","owner":"obsidianjulua","description":"Julia build system","archived":false,"fork":false,"pushed_at":"2025-12-13T15:09:15.000Z","size":11110,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-12-14T07:31:40.535Z","etag":null,"topics":["build-system","cplusplus","julia","llvm-ir","project-management"],"latest_commit_sha":null,"homepage":"https://github.com/obsidianjulua/RepliBuild.jl","language":"Julia","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/obsidianjulua.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-10T22:52:18.000Z","updated_at":"2025-12-13T15:09:18.000Z","dependencies_parsed_at":null,"dependency_job_id":"422adfc5-ab2c-4fa2-b72e-6e333a4138c4","html_url":"https://github.com/obsidianjulua/RepliBuild.jl","commit_stats":null,"previous_names":["obsidianjulua/replibuild.jl"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/obsidianjulua/RepliBuild.jl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/obsidianjulua%2FRepliBuild.jl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/obsidianjulua%2FRepliBuild.jl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/obsidianjulua%2FRepliBuild.jl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/obsidianjulua%2FRepliBuild.jl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/obsidianjulua","download_url":"https://codeload.github.com/obsidianjulua/RepliBuild.jl/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/obsidianjulua%2FRepliBuild.jl/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28057674,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-12-26T02:00:06.189Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["build-system","cplusplus","julia","llvm-ir","project-management"],"created_at":"2025-10-21T12:01:30.851Z","updated_at":"2026-04-11T10:08:25.833Z","avatar_url":"https://github.com/obsidianjulua.png","language":"Julia","funding_links":[],"categories":[],"sub_categories":[],"readme":"# RepliBuild.jl\n\n[![Documentation](https://img.shields.io/badge/docs-dev-blue.svg)](https://obsidianjulua.github.io/RepliBuild.jl/dev/)\n[![Julia 1.10+](https://img.shields.io/badge/julia-1.10+-9558B2?logo=julia)](https://julialang.org)\n[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)\n\nRepliBuild is an ABI-aware C/C++ compiler bridge for Julia. It compiles C/C++ source through an LLVM/MLIR pipeline, introspects DWARF debug metadata and symbol tables, and emits type-safe Julia bindings with correct struct layout, enum definitions, and calling conventions — no hand-written bindings, no header annotations, no build system integration required.\n\nThe core idea: point RepliBuild at a C/C++ project, and it produces a Julia module where every function, struct, enum, union, template instantiation, and virtual method is callable with correct ABI semantics. Functions are automatically routed to one of three calling tiers based on ABI complexity.\n\n## Three-Tier Dispatch\n\nEvery function signature is analyzed against DWARF metadata at wrap time. The wrapper generator selects the most efficient calling mechanism that can handle the function's ABI requirements:\n\n| Tier | Mechanism | When Selected | Overhead |\n|------|-----------|---------------|----------|\n| **1** | `Base.llvmcall` with LTO bitcode | POD args, scalar/pointer return, LTO bitcode available | Zero — C/C++ IR merges into Julia's JIT, enabling cross-language inlining and vectorization |\n| **2** | MLIR AOT/JIT thunks via `libJLCS.so` | Packed structs, unions, large struct return, virtual dispatch, exception-throwing functions | One thunk indirection — compiled once, cached with lock-free read path |\n| **3** | `ccall` | Fallback when bitcode is unavailable | Standard FFI overhead |\n\n**Tier 1** — LTO artifacts are LLVM bitcode assembled by `Clang_unified_jll` to guarantee version-matched IR with Julia's internal LLVM. The C/C++ IR merges directly into Julia's JIT pipeline. This enables cross-language optimizations: inlining, SIMD vectorization, and compatibility with AD tools like Enzyme.jl. Enabled by default for pure-C projects.\n\n**Tier 2** — A custom MLIR dialect (`jlcs`) handles ABI marshalling that `llvmcall` and `ccall` cannot express. This includes packed struct returns, C++ virtual dispatch through vtables, scoped RAII lifetime management, and C++ exception catching. Thunks can be JIT-compiled on first call or pre-compiled to a `_thunks.so` at build time (`aot_thunks = true`).\n\n**Tier 3** — Direct `ccall` with zero setup. The unconditional fallback.\n\n## The JLCS MLIR Dialect\n\nRepliBuild includes a custom MLIR dialect — **JLCS** (Julia C-Struct) — that models C/C++ ABI semantics as first-class IR operations. The dialect is defined in TableGen (`src/mlir/`), compiled to `libJLCS.so`, and accessed from Julia via C API wrappers.\n\n### Types\n\n| Type | Mnemonic | Purpose |\n|------|----------|---------|\n| `CStructType` | `!jlcs.c_struct\u003c\"Name\", [types], [offsets], packed=bool\u003e` | C-ABI struct with explicit field types, byte offsets, and packing. The core type for representing any C/C++ struct with layout fidelity. |\n| `ArrayViewType` | `!jlcs.array_view\u003celement, rank\u003e` | Universal strided array descriptor — maps to Julia `Array{T,N}`, NumPy ndarray, and C++ strided arrays for zero-copy interop. |\n\n### Operations\n\n**Metadata and field access:**\n\n| Op | Mnemonic | Description |\n|----|----------|-------------|\n| `TypeInfoOp` | `jlcs.type_info` | Declares a C struct type with its Julia name, MLIR struct type, C++ base class (inheritance), and destructor symbol. Module-level metadata. |\n| `GetFieldOp` | `jlcs.get_field` | Reads a struct field at a byte offset. Lowers to GEP + load. |\n| `SetFieldOp` | `jlcs.set_field` | Writes a struct field at a byte offset. Lowers to GEP + store. |\n\n**Function calls:**\n\n| Op | Mnemonic | Description |\n|----|----------|-------------|\n| `VirtualCallOp` | `jlcs.vcall` | Calls a C++ virtual method: loads the vtable pointer from the object, indexes the function pointer by slot, and calls with the object as `this`. |\n| `FFECallOp` | `jlcs.ffe_call` | Foreign function execution — calls an external C/C++ function with ABI-correct argument marshalling. |\n| `TryCallOp` | `jlcs.try_call` | Like `ffe_call` but emits `invoke` + landing pad to catch C++ exceptions. On catch, stores the `what()` message in a thread-local buffer and returns a sentinel. Julia checks and throws `CxxException`. |\n\n**Array access:**\n\n| Op | Mnemonic | Description |\n|----|----------|-------------|\n| `LoadArrayElementOp` | `jlcs.load_array_element` | Loads from a strided multi-dimensional array view using linearized offset computation. |\n| `StoreArrayElementOp` | `jlcs.store_array_element` | Stores to a strided multi-dimensional array view. |\n\n**RAII (C++ object lifetime):**\n\n| Op | Mnemonic | Description |\n|----|----------|-------------|\n| `ConstructorCallOp` | `jlcs.ctor_call` | Calls a C++ constructor with `this` pointer + parameters. |\n| `DestructorCallOp` | `jlcs.dtor_call` | Calls a C++ destructor on an object pointer. |\n| `ScopeOp` | `jlcs.scope` | Region-based RAII scope — takes managed object pointers and destructor symbols. Body ops are inlined during lowering; destructors are emitted in reverse order (C++ semantics). |\n| `YieldOp` | `jlcs.yield` | Terminator for `jlcs.scope` regions. |\n\n### Example IR\n\n```mlir\n// Declare a packed struct with field offsets\njlcs.type_info \"Vec3\", !jlcs.c_struct\u003c\"Vec3\", [f32, f32, f32], [0, 4, 8], packed = false\u003e, \"\", \"\"\n\n// RAII scope: construct, use, auto-destruct\n%alloca = llvm.alloca 1 x !llvm.struct\u003c(i32, i32)\u003e : (i64) -\u003e !llvm.ptr\njlcs.scope(%alloca : !llvm.ptr) dtors([@_ZN4BaseD1Ev]) {\n  jlcs.ctor_call @_ZN4BaseC1Ei(%alloca, %val) : (!llvm.ptr, i32) -\u003e ()\n  %result = jlcs.vcall @Base::area(%alloca) { vtable_offset = 0, slot = 2 } : (!llvm.ptr) -\u003e f64\n  jlcs.yield\n}\n\n// Exception-safe call\n%ret = jlcs.try_call %arg0 { callee = @_Z12might_throwi } : (i32) -\u003e i32\n```\n\nAll JLCS ops lower to LLVM IR via `JLCSPasses.cpp`. The lowered IR is executed either through the MLIR JIT engine (on-demand) or compiled ahead-of-time into `_thunks.so`.\n\n## What Gets Wrapped\n\nRepliBuild introspects DWARF debug metadata and symbol tables to generate bindings for:\n\n- **Structs** — Correct field order, alignment padding, topological dependency sort, `Ptr{X}` soft-dependency handling for circular references\n- **Enums** — `@enum` with correct underlying types; Clang.jl AST walker handles `enum class`, hex values, namespaces\n- **Unions** — `NTuple{N,UInt8}` backing with typed getter/setter accessors\n- **Bitfields** — Bit-level extraction from packed representations\n- **Function pointers** — DWARF signature parsing to `@cfunction`-compatible type strings\n- **Variadic functions** — Typed overloads declared in `[wrap.varargs]`\n- **Multi-level pointers / references** — `T**` to `Ptr{Ptr{T}}`, `T\u0026` to `Ref{T}`\n- **C++ virtual methods** — MLIR JIT thunks or static AOT thunks via vtable slot dispatch\n- **C++ exceptions** — Functions that throw are routed through `jlcs.try_call`; exceptions surface as `CxxException` in Julia\n- **Templates** — Declare `templates = [\"std::vector\u003cint\u003e\"]`; RepliBuild forces Clang to emit DWARF for those instantiations\n- **STL containers** — `CppVector{T}`, `CppString`, `CppMap{K,V}` wrappers with Julia `AbstractArray`/`AbstractDict` interfaces\n- **Idiomatic wrappers** — Factory/destructor pairs clustered by class name into `mutable struct` with GC finalizers and multiple-dispatch method proxies\n- **Global variables** — `cglobal` accessors\n\n## Pipeline\n\n```\nC/C++ Source + [dependencies]\n    |\n    v\nDependencyResolver   -- clone/update git deps, filter excludes, inject into compile graph\n    |\n    v\nDiscovery            -- scan files, resolve #include graph, emit replibuild.toml\n    |\n    v\nCompiler             -- Clang/clang++ -\u003e per-file LLVM IR; incremental mtime + project-hash cache\n    |\n    v\nLinker               -- llvm-link + llvm-opt -\u003e .so/.dylib + _lto.bc (Tier 1) + _thunks.so (Tier 2)\n    |\n    v\nDWARFParser          -- llvm-dwarfdump + nm -\u003e ClassInfo / VtableInfo / MemberInfo structs\n    |\n    v\nDispatchLogic        -- per-function tier routing: is_ccall_safe(), is_c_lto_safe()\n    |\n    v\nWrapper              -- DWARF + symbols -\u003e Julia module (C and C++ generators are independent)\n    |\n    v\nJITManager           -- on-demand: JLCSIRGenerator -\u003e MLIR JLCS IR -\u003e MLIRNative JIT -\u003e thunk cache\n```\n\n### Caching\n\nTwo independent layers ensure sub-second rebuilds:\n\n- **Per-file IR cache** (`.replibuild_cache/`) — mtime-based, skips individual source files whose IR is current.\n- **Project content hash** — SHA256 of `replibuild.toml` + all source/header contents + git HEAD. If the hash matches, `build()` exits without invoking any compiler.\n\n### Key Modules\n\n| Module | Role |\n|--------|------|\n| `Compiler.jl` | Core build engine: per-file LLVM IR, incremental cache, parallel compilation, LTO linking, bitcode assembly |\n| `DWARFParser.jl` | Parses `llvm-dwarfdump` + `nm` output into structured `ClassInfo`, `VtableInfo`, `MemberInfo` |\n| `Wrapper/Generator.jl` | Top-level `wrap_library()` — routes to C or C++ generator based on language config |\n| `Wrapper/C/GeneratorC.jl` | C wrapper generator: struct packing, enums, bitfields, unions, varargs |\n| `Wrapper/Cpp/GeneratorCpp.jl` | C++ wrapper generator: templates, STL, vtable thunks, managed types |\n| `Wrapper/DispatchLogic.jl` | Per-function tier routing decisions (`is_ccall_safe`, `is_c_lto_safe`) |\n| `JLCSIRGenerator.jl` | DWARF types to MLIR JLCS IR generation |\n| `MLIRNative.jl` | Julia `ccall` bindings to `libJLCS.so` C API |\n| `JITManager.jl` | MLIR JIT lifecycle, atomic copy-on-write symbol cache, `CxxException` handling |\n| `Discovery.jl` | Project scanning, `#include` graph resolution, `replibuild.toml` generation |\n| `PackageRegistry.jl` | Content-addressed build cache at `~/.replibuild/`, `use()`/`register()` API |\n\n## Quick Start\n\n```julia\nusing RepliBuild\n\n# One call: scan, compile, and wrap a C/C++ project\nRepliBuild.discover(\"path/to/project\", build=true, wrap=true)\n\n# Load the generated module\ninclude(\"path/to/project/julia/MyProject.jl\")\nusing .MyProject\n```\n\nOr step by step:\n\n```julia\ntoml = RepliBuild.discover(\"path/to/project\")  # generates replibuild.toml\nRepliBuild.build(toml)                          # Clang -\u003e LLVM IR -\u003e .so + DWARF\nRepliBuild.wrap(toml)                           # DWARF -\u003e Julia module\n```\n\n### Package Registry\n\n```julia\nRepliBuild.register(\"path/to/project/replibuild.toml\")  # one-time registration\nLua = RepliBuild.use(\"lua\")                              # build + wrap + load, cached\nLua.luaL_newstate()\n\nRepliBuild.search(\"xml\")                         # search the RepliBuild Hub\nRepliBuild.list_registry()                       # print all registered packages\n```\n\n## Configuration\n\n`replibuild.toml` is generated by `discover()` and is hand-editable:\n\n```toml\n[project]\nname = \"MyProject\"\n\n[compile]\nflags    = [\"-std=c++17\", \"-fPIC\", \"-O3\"]\nparallel = true\n\n[link]\noptimization_level = \"3\"\nenable_lto         = false   # true -\u003e emit _lto.bc for Base.llvmcall (Tier 1)\n\n[binary]\ntype           = \"shared\"    # \"shared\" | \"static\" | \"executable\"\nstrip_symbols  = false\n\n[wrap]\nlanguage     = \"cpp\"         # \"c\" | \"cpp\" (auto-detected by discover())\nuse_clang_jl = true\naot_thunks   = false         # true -\u003e pre-compile MLIR thunks to _thunks.so (Tier 2)\n\n[wrap.varargs]\nprintf = [[\"Cstring\", \"Cint\"], [\"Cstring\", \"Cdouble\"]]\n\n[types]\nstrictness             = \"warn\"   # \"strict\" | \"warn\" | \"permissive\"\nallow_unknown_structs  = true\nallow_function_pointers = true\ntemplates              = [\"std::vector\u003cint\u003e\"]\ntemplate_headers       = [\"\u003cvector\u003e\"]\n\n[cache]\nenabled   = true\ndirectory = \".replibuild_cache\"\n\n[dependencies.cjson]\ntype    = \"git\"\nurl     = \"https://github.com/DaveGamble/cJSON\"\ntag     = \"v1.7.18\"\nexclude = [\"test\", \"fuzzing\", \"CMakeLists.txt\"]\n```\n\n## Requirements\n\n- Julia 1.10+\n- LLVM 21+ and Clang (system install; auto-detected, JLL fallback for Tier 1)\n- CMake 3.20+ and `mlir-tblgen` (Tier 2 only)\n\nRun `RepliBuild.check_environment()` to verify which tiers are available.\n\n## Test Coverage\n\n| Project | Description |\n|---------|-------------|\n| Lua 5.4.6 | Full VM + stdlib: state management, stack ops, callbacks, coroutines |\n| SQLite 3.49.1 | 261 K-line C API: varargs, opaque pointer lifecycle |\n| Duktape 2.7.0 | 101 K-line JS engine amalgamation: monolithic compile, stack round-trips |\n| cJSON | Multi-file C library: git dependency resolution |\n| Stress test | Vectors, matrices, numerics, vtable dispatch, RAII, MLIR/AOT |\n| Callback test | Bidirectional FFI with C++ exception propagation |\n| STL test | `CppVector`, `CppString`, `CppMap` lifecycle and interop |\n\n## Documentation\n\n- [Why RepliBuild](https://obsidianjulua.github.io/RepliBuild.jl/dev/why-replibuild/) — What it solves, comparison to alternatives\n- [User Guide](https://obsidianjulua.github.io/RepliBuild.jl/dev/guide/)\n- [Configuration Reference](https://obsidianjulua.github.io/RepliBuild.jl/dev/config/)\n- [Introspection Tools](https://obsidianjulua.github.io/RepliBuild.jl/dev/introspect/)\n- [MLIR / JLCS Dialect](https://obsidianjulua.github.io/RepliBuild.jl/dev/mlir/)\n- [Architecture](https://obsidianjulua.github.io/RepliBuild.jl/dev/architecture/)\n- [Changelog](CHANGELOG.md)\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fobsidianjulua%2Freplibuild.jl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fobsidianjulua%2Freplibuild.jl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fobsidianjulua%2Freplibuild.jl/lists"}