{"id":51910569,"url":"https://github.com/wabiverse/lattice","last_synced_at":"2026-07-27T04:00:51.653Z","repository":{"id":369243417,"uuid":"1289025956","full_name":"wabiverse/Lattice","owner":"wabiverse","description":"A Swift-native runtime store for real-time scenes with Fabric-style performance gains.","archived":false,"fork":false,"pushed_at":"2026-07-15T00:02:56.000Z","size":473,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-07-15T00:25:37.296Z","etag":null,"topics":["architype","columnar-storage","computer-graphics","ecs","gpu","gpu-acceleration","graphics","metal","nvidia","omniverse","openusd","real-time","simulation","swift","usd"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wabiverse.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-07-04T08:47:27.000Z","updated_at":"2026-07-15T00:03:00.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/wabiverse/Lattice","commit_stats":null,"previous_names":["wabiverse/lattice"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/wabiverse/Lattice","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wabiverse%2FLattice","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wabiverse%2FLattice/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wabiverse%2FLattice/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wabiverse%2FLattice/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wabiverse","download_url":"https://codeload.github.com/wabiverse/Lattice/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wabiverse%2FLattice/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35935763,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-07-20T02:08:10.276Z","status":"online","status_checked_at":"2026-07-27T02:00:06.776Z","response_time":101,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["architype","columnar-storage","computer-graphics","ecs","gpu","gpu-acceleration","graphics","metal","nvidia","omniverse","openusd","real-time","simulation","swift","usd"],"created_at":"2026-07-27T04:00:50.858Z","updated_at":"2026-07-27T04:00:51.644Z","avatar_url":"https://github.com/wabiverse.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Lattice\n\n\u003cp align=\"center\"\u003e\n  \u003cimg width=\"640\" height=\"474\" alt=\"target\" src=\"https://github.com/user-attachments/assets/b8a487a3-6aa8-40f3-9fcc-a223e2e08703\" /\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003csub\u003e\n    100,000 cubes moving at 60 fps. A Metal kernel writes every transform, a\n    Hydra scene index reads them straight out of the store, and the \u003ccode\u003eUsdStage\u003c/code\u003e\n    is never touched. Lattice's share of the 16.64 ms frame is 0.53 ms.\n  \u003c/sub\u003e\n\u003c/p\u003e\n\nA Swift-native, open-source runtime data store for real-time scenes -\nthe same problem [NVIDIA's Fabric/USDRT](https://docs.omniverse.nvidia.com/kit/docs/usdrt.scenegraph/latest/usd_fabric_usdrt.html)\nsolve inside Omniverse, built as its own thing rather than a port of that API.\n\nLattice is not a scene graph, not a composition engine, and not tied to\nUSD. It's a small archetype-based store: entities, columns of component\ndata laid out for cache-friendly bulk iteration, and cheap structural moves\nwhen an entity's component set changes. It's meant to sit *next to*\nwhatever owns your authoritative scene description - a `UsdStage` via\n`LatticeUSD`, or nothing at all - the same way Fabric sits next to a stage\nwithout replacing it.\n\n## Why this exists\n\nFabric gives Omniverse a place to read and write scene data at\nper-frame rates without paying USD composition and `TfNotice` overhead.\nUSDRT is a USD-shaped API on top of it. Neither is open source - only the\nUSDRT API layer ships as source/binary, and Fabric itself is developed\ninside Kit. There isn't an existing open equivalent for people who want\nFabric-style performance in their own Swift engine without Omniverse.\n\nLattice borrows the two ideas that actually matter from Fabric's design -\nbucketed, columnar storage, and cheap change tracking instead of per-value\nnotifications - and otherwise takes a completely Swift-native shape: no\n`usdrt::UsdStage`-alike API, no attempt at source compatibility with anything\nNVIDIA ships. What Fabric does with CUDA and a C++ Kit runtime, Lattice does\nwith Swift value types, contiguous columns, `MTLBuffer`-backed storage on\nunified memory, and Swift's own concurrency for parallel iteration.\n\n## Package layout\n\n- **`LatticeCore`** - the core. Entities, archetypes, columns, queries, change\n  tracking. No platform-specific or USD-specific code lives here; it builds\n  and tests anywhere Swift runs.\n- **`LatticeMetal`** - `MetalBackedColumn\u003cT\u003e`, a column backed by an\n  `MTLBuffer` instead of a Swift array, wired into the store through a\n  per-component factory. On Apple Silicon's unified memory, writes here are\n  immediately visible to the GPU with no upload step:\n  `store.register(Particle.self) { MetalBackedColumn\u003cParticle\u003e(device: device) }`.\n- **`LatticeUSD`** - a thin adapter (`USDStageSourceRepresentable`) that lets any\n  USD binding populate a `LatticeStore`, without Lattice depending on that\n  binding's concrete API. It also holds the two objects the Hydra scene indices\n  read live transforms out of: `LatticeXformSource` for per-prim transforms and\n  `LatticeInstanceSource` for instance arrays.\n- **`LatticeOverlays`** - small C++ helpers for the bits of OpenUSD that Swift\n  can't reach directly yet, mainly zero-copy `VtArray` access.\n- **`lattice`** - the C++ side of the Hydra integration: the two scene indices,\n  and the small C bridge that registers them with Hydra. See [Live in Hydra](#live-in-hydra).\n- **`LatticeDemo`** - `swift run -c release LatticeDemo`: spawns 100k entities\n  with a `Transform`/`Velocity` pair and integrates them for 120 frames across\n  the serial, parallel, and Metal GPU paths, then repeats the whole run over a\n  store populated from a real `UsdStage`. See [Benchmarks](#benchmarks).\n- **`LatticeHydraDemo`** - `swift run -c release LatticeHydraDemo`: the same\n  store, this time driving a live Hydra viewport with 100k moving cubes. See\n  [Live in Hydra](#live-in-hydra).\n\n## Benchmarks\n\n100,000 entities, 120 frames, one `Transform`/`Velocity` pair each, integrated\nwith a deliberately compute-bound per-entity kernel (32 iterations of trig +\n`sqrt` - the scenario where throughput, not memory or dispatch overhead, decides).\nEvery path runs identical math, the GPU folds 30 frames into each dispatch to\namortize command-buffer cost. Release build, base **2026 Apple MacBook Air M5**\n(10-core, unified memory).\n\n| Path | Per-frame | vs serial CPU | vs parallel CPU |\n| :--- | ---: | ---: | ---: |\n| CPU - serial | 33.8 ms | 1× | - |\n| CPU - parallel | 6.35 ms | 5.3× | 1× |\n| **GPU - Metal, unified memory** | **0.13 ms** | **254×** | **48×** |\n\nDriven end-to-end through `LatticeUSD` - 100k prims authored into a `UsdStage`,\nloaded, and mirrored into the store via `USDPopulationSync` - the frame loop hits\nthe same numbers (**228×** serial, **45×** parallel), with the one-time USD-\u003estore\npopulation costing ~1.1 s.\n\n\u003e [!NOTE]\n\u003e This is illustrative of the architecture, not a head-to-head benchmark\n\u003e against Fabric/USDRT (which use different hardware, kernels, and APIs).\n\u003e The core takeaway is that a Swift-native, columnar `MTLBuffer` on\n\u003e unified memory achieves GPU-throughput territory on the exact same\n\u003e per-frame simulation patterns Fabric targets - entirely bypassing the\n\u003e need for a separate upload step, CUDA, or the Kit runtime.\n\n## Live in Hydra\n\n`LatticeHydraDemo` runs the store behind a live Hydra viewport.\n\nThe cubes are written into a `UsdStage` once, at startup, and after that the\nstage is never touched again. Every frame a Metal kernel rewrites all 100,000\ntransforms in a `MetalBackedColumn`, and a scene index hands those to Hydra\nwhen it asks for a prim - so Hydra reads the store, not the stage.\n\nThe stage still holds the scene as authored, and the motion lives somewhere\nthat can keep up with a frame. That's the same division Fabric uses inside\nOmniverse, done here as a Hydra scene index.\n\n### The scene shape decides the frame time, not the store\n\nThe demo can run the same store and the same kernel two ways. The only thing\nthat changes is how many prims Hydra has to be told about, and that turns out\nto be what decides the frame time. Release build, same base **2026 Apple\nMacBook Air M5** (10-core, unified memory) as the benchmarks above, 100k cubes,\n`Ripple` kernel, Storm:\n\n| 100k cubes | xform compute | dirty + notify | frame |\n| :--- | ---: | ---: | ---: |\n| `--per-prim` - one `Cube` prim each | 0.35 ms | 56.23 ms | 1033 ms |\n| instancer - one `UsdGeomPointInstancer` | 0.43 ms | **0.07 ms** | **26 ms** |\n\nWith one prim per cube, Hydra re-syncs 100,000 prims every frame, and we have\nto hand it 100,000 dirty paths to make that happen. The instancer replaces\nthree arrays on a single prim instead, so there's one dirty path no matter how\nmany cubes there are. That's roughly 800× less time spent notifying and 40×\nless per frame, with the store doing exactly the same work either way.\n\nLattice costs under half a millisecond in both. Everything else in the frame\ndwarfs it.\n\n\u003e [!NOTE]\n\u003e These are read off the demo's HUD, not a proper benchmark harness, and\n\u003e `frame` includes Storm drawing and presenting. The gap between the two rows\n\u003e is the part that carries over to other machines - it comes from prim counts,\n\u003e not from this laptop.\n\n### The frame contract\n\nHydra calls `GetPrim()` from several threads at once, so writing to the store\nwhile it reads would corrupt it. The frame is split into two phases, and\n`LatticeFramePhase` asserts on the split in debug builds:\n\n```\nmutate -\u003e advanceChangeTick() -\u003e sceneIndex.Tick() -\u003e beginReadPhase()\n       -\u003e Hydra pulls GetPrim() -\u003e endReadPhase()\n```\n\n`Hydra.FrameDelegate` provides those two hooks. By the time the read phase\nopens, everything Hydra is about to ask for has been written and marked dirty.\n\n### Motion fields, switchable live\n\nFive kernels. Each one works out where a cube should be from its home position\nand the clock, and nothing else - nothing carried over from last frame, nothing\nshared between cubes. That's what makes them easy to run in parallel, and it's\nalso why you can switch between them while it's running with nothing to reset.\nThey're all compiled at startup, so switching is instant:\n\n| Kernel | Motion | Per-instance cost |\n| :--- | :--- | :--- |\n| Ripple | spherical wave from the centre | ~20 flops |\n| Galaxy | differential rotation winds a grid into spiral arms | ~40 flops |\n| Curl Noise | divergence-free curl of an fbm vector potential | 18 fbm, 3 octaves |\n| Lorenz | the attractor, re-integrated from home every frame | 128 Euler steps |\n| Mandelbulb | distance estimation with an animated exponent | 16 iters, `pow`/`acos`/`atan2` |\n\nClicking down that list is the fun part. `xform compute` goes up roughly\ntenfold from top to bottom, and the frame time barely moves.\n\n```pwsh\nexport SWIFTUSD_BUILD_FROM_SOURCE=1\n\nswift run -c release LatticeHydraDemo                 # instancer + GPU, 100k\nswift run -c release LatticeHydraDemo --count 250000  # more\nswift run -c release LatticeHydraDemo --per-prim      # the per-prim comparison\nswift run -c release LatticeHydraDemo --cpu           # parallel-CPU path (ripple only)\n```\n\n## Core concepts\n\n- **`LatticeEntity`** - a dense index/generation handle. No data, no path,\n  no name. Just an identity.\n- **`LatticeComponent`** - marker protocol for a storable value type.\n- **`Archetype`** - a bucket of entities sharing exactly the same component\n  types, holding one densely packed column per type.\n- **`LatticeStore`** - owns every archetype, and is the only place that\n  moves an entity between archetypes when you `set`/`remove` a component\n  type it didn't previously have.\n- **`Query1\u003cA\u003e` ... `Query4\u003cA, B, C, D\u003e`** - read or mutate matching entities by\n  iterating archetype columns directly, not by looking entities up one at a\n  time. Iteration hands the closure contiguous buffers so the loop vectorizes;\n  `forEachMutatingFirstParallel` fans the same work across cores.\n- **`mutationGeneration(of:)`** - a coarse \"did anything of this component\n  type change\" counter, standing in for `TfNotice`.\n- **`currentTick` / `forEachChanged(since:)`** - per-row change detection:\n  every write stamps the row with a monotonic tick, so a system can touch only\n  the entities whose component changed since it last ran (Bevy-style), the\n  fine-grained counterpart to `mutationGeneration(of:)`.\n\n```swift\nlet store = LatticeStore()\n\n// No registration step needed - set/spawn register a component on first use.\nlet entity = store.spawn(\n    Transform(x: 0, y: 0, z: 0),\n    Velocity(dx: 1, dy: 0, dz: 0)\n)\n\n// Bulk-iterate matching entities over contiguous columns.\nstore.query(Transform.self, Velocity.self).forEachMutatingFirst { _, transform, velocity in\n    transform.x += velocity.dx\n}\n\n// The same loop, fanned out across cores (data-parallel, single-writer for structure):\nstore.query(Transform.self, Velocity.self).forEachMutatingFirstParallel { transform, velocity in\n    transform.x += velocity.dx\n}\n```\n\n## Concurrency model\n\nFabric exists so many systems can read and write scene data per frame without\nserializing on composition. Lattice takes the same position with a clear,\nenforceable contract:\n\n- **Value mutation is data-parallel.** `forEachMutatingFirstParallel` splits an\n  archetype's rows into contiguous batches across the global concurrent queue.\n  Each worker owns a disjoint row range, so mutating one component while reading\n  others needs no locking. This is the embarrassingly-parallel per-frame\n  simulation path.\n- **Structural change is single-writer.** `spawn`, `despawn`, `set`-that-adds-a\n  -type, and `remove` move entities between archetypes and mutate shared\n  indices. They must not run concurrently with a query. Run structural edits\n  between parallel passes, not during them - the same discipline Fabric's\n  bucket model requires.\n- **Systems run in parallel across disjoint access sets.** A `LatticeSystem`\n  declares the component types it reads and writes; `LatticeScheduler` groups\n  non-conflicting systems into waves that run concurrently, ordering only the\n  pairs that actually conflict (write-write, or read-vs-write on the same type).\n  This is the across-systems complement to the within-a-query parallelism above\n  - the scheduling model Fabric uses to fill a frame across cores.\n\n```swift\nlet scheduler = LatticeScheduler()\nscheduler.add(LatticeSystem(\"integrate\", reads: [Velocity.self], writes: [Transform.self]) { store in\n    store.query(Transform.self, Velocity.self).forEachMutatingFirst { _, t, v in t.x += v.dx }\n})\nscheduler.add(LatticeSystem(\"regen\", writes: [Health.self]) { store in\n    store.query(Health.self).forEachMutating { _, h in h.hp += 1 }\n})\nscheduler.run(on: store)   // integrate and regen touch different types -\u003e one concurrent wave\n```\n\n## What's implemented\n\n- **Optional registration.** `set`/`spawn` register a component the first time\n  they see it; there's no mandatory startup step. `register(_:columnFactory:)`\n  remains, now solely to choose a component's backing storage.\n- **Selectable column backing.** Columns are created through a per-component\n  factory, so a type can live in a plain array (`TypedColumn`) or directly in\n  an `MTLBuffer` (`MetalBackedColumn`) - queries drive either transparently\n  through the `TypedColumnStorage` protocol. `MetalBackedColumn` also exposes a\n  `bufferGeneration` so a renderer knows when a growth reallocation invalidated\n  the buffer it had bound.\n- **Two-level change detection.** The coarse `mutationGeneration(of:)` answers\n  \"did any `T` change\"; per-row change ticks (`currentTick`,\n  `forEachChanged(since:)`) answer \"*which* entities' `T` changed\", and ticks\n  are preserved across archetype moves so an unrelated add/remove never looks\n  like a value edit.\n- **Queries up to arity four**, with vectorizable contiguous iteration, the\n  parallel mutation path above, and `excluding:` negative filters\n  (`store.query(Transform.self, excluding: Hidden.self)`).\n- **A system scheduler** (`LatticeSystem` / `LatticeScheduler`) that runs\n  systems concurrently when their declared read/write sets are disjoint, and\n  orders the conflicting pairs.\n- **USD population, synchronization, *and* write-back - the full loop.**\n  `syncAll()` does one-time population; `syncIncremental()` diffs the stage's\n  current prim set against the last-seen set and spawns/despawns only the delta;\n  `writeBackChanged(...)` authors component values back onto stage attributes,\n  touching only the rows that changed since a given tick (via the same per-row\n  change ticks). That's USDRT's population-vs-synchronization split plus the\n  return path. `USDStageSource` is a concrete `USDStageSourceRepresentable`\n  backed by a real `UsdStage` via `wabiverse/swift-usd`.\n- **A live Hydra path.** Two scene indices - `LatticeHydraSceneIndex` for\n  per-prim transforms, `LatticeInstancerSceneIndex` for instance arrays - feed\n  Hydra from the store without touching the stage. The read/write phase split\n  is asserted in debug builds, not just written down.\n  See [Live in Hydra](#live-in-hydra).\n\n## Roadmap\n\nWhat's genuinely still ahead, in rough priority:\n\n- **Query arity beyond four** via parameter packs rather than more overloads,\n  and **optional components** in a query (visit entities with `A`, and `B` if\n  present).\n- **Structural commands buffered from within systems** - let a scheduled system\n  record spawn/despawn/add/remove requests that the store applies at the wave\n  barrier, so structural change composes with the parallel scheduler instead of\n  sitting strictly between runs.\n- **Write-back driven by an execution graph.** `writeBackChanged` already closes\n  the store-\u003estage loop by change tick; wiring it to `ExecUsdSystem`'s\n  invalidation graph would let recompute *and* write-back share the one signal\n  that already knows what's dirty.\n- **Zero-copy arrays across the scene index boundary.** `VtArray`'s foreign data\n  source hook could be utilized to reference the `MTLBuffer` directly, making the\n  whole path allocation-free.\n\n## Integrating with `wabiverse/swift-usd`\n\n`LatticeUSD` depends on `OpenUSDKit` from `wabiverse/swift-usd` and ships\n`USDStageSource`, a concrete `USDStageSourceRepresentable` backed by a\nreal `UsdStage`: it traverses the composed stage for prim paths and reads\nresolved attribute values, mapping USD value types to `LatticeUSDValue`.\n\n```pwsh\ngit clone https://github.com/wabiverse/Lattice.git\ncd Lattice\n\n# since apple/SwiftUsd does not yet expose zero-copy\n# read-only buffer access via VtArray::cdata(), or\n# single-copy construction from a contiguous buffer\n# issue: https://github.com/apple/SwiftUsd/issues/34\n# this will require building OpenUSD from source with\n# the following environment variable:\nexport SWIFTUSD_BUILD_FROM_SOURCE=1\n\n# benchmark your own production scene\nswift run -c release LatticeDemo --usd /path/to/stage.usda\n```\n\n```swift\nlet source = USDStageSource(openingStageAt: \"scene.usd\")\nlet sync = USDPopulationSync(store: store, paths: paths, source: source)\nsync.syncAll()   // one-time population\nsync.populate(Transform.self, from: \"xformOp:translate\") { value in\n    guard case let .float3(x, y, z) = value else { return nil }\n    return Transform(x: x, y: y, z: z)\n}\n// Later, each frame the stage's prim set can change:\nsync.syncIncremental()   // spawns/despawns only the delta\n```\n\nIf we extend `ExecUsdSystem` to write its computed values (posed\ntransforms, bounds) into a `LatticeStore` - rather than only handing\nback an `ExecUsdCacheView` - we'd close a real gap. OpenExec does\nnot currently author values back into scene description, it only\nobserves the stage and returns results via a cache view. Fabric\ngets store-plus-invalidation writes today, but through OmniGraph,\nnot OpenExec. I haven't found anything public that pairs Pixar's\nown schema-aware execution system with a Fabric-style store - if\nthat exists already, I'd love to know - but if not, this would\ngive OpenExec's invalidation graph a real place to write.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwabiverse%2Flattice","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwabiverse%2Flattice","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwabiverse%2Flattice/lists"}