{"id":22572267,"url":"https://github.com/lokua/lattice","last_synced_at":"2025-04-15T03:04:54.078Z","repository":{"id":265765159,"uuid":"896368797","full_name":"Lokua/lattice","owner":"Lokua","description":"A hobbyist project exploring generative art while learning Rust and Nannou.","archived":false,"fork":false,"pushed_at":"2025-04-15T02:11:32.000Z","size":5901,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-15T03:04:50.154Z","etag":null,"topics":["audio","creative-coding","framework","midi","osc","shaders"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Lokua.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-30T07:06:48.000Z","updated_at":"2025-04-15T02:09:54.000Z","dependencies_parsed_at":"2024-12-26T11:27:20.847Z","dependency_job_id":"925d6279-d1ff-412f-91be-2a68c8705504","html_url":"https://github.com/Lokua/lattice","commit_stats":null,"previous_names":["lokua/lattice"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Lokua%2Flattice","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Lokua%2Flattice/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Lokua%2Flattice/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Lokua%2Flattice/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Lokua","download_url":"https://codeload.github.com/Lokua/lattice/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248997083,"owners_count":21195799,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["audio","creative-coding","framework","midi","osc","shaders"],"created_at":"2024-12-08T02:08:41.764Z","updated_at":"2025-04-15T03:04:54.071Z","avatar_url":"https://github.com/Lokua.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# lattice\n\nA hobbyist project exploring generative art while learning Rust and\n[nannou][nannou]. Stuff like this:\n\n\u003cimg \n  src=\"./images/displacement_2-627iz.png\" \n  alt=\"displacement_2-627iz\" \n  width=\"90%\"\n/\u003e\n\n\u003c!-- \u003cdiv\u003e\n   \u003cimg src=\"./images/displacement_2-tm8s9.png\" alt=\"displacement_2-tm8s9\" width=\"30%\"\u003e\n   \u003cimg src=\"./images/sierpinski_triangle-0x9az.png\" alt=\"sierpinski_triangle-0x9az\" width=\"30%\"\u003e\n   \u003cimg src=\"./images/displacement_2-vnh7y.png\" alt=\"displacement_2-vnh7y\" width=\"30%\"\u003e\n\u003c/div\u003e\n\u003cdiv\u003e\n   \u003cimg src=\"./images/vertical-1nhw3.png\" alt=\"vertical-1nhw3\" width=\"30%\"\u003e\n   \u003cimg src=\"./images/genuary_2-4psv0.png\" alt=\"genuary_2-4psv0\" width=\"30%\"\u003e\n   \u003cimg src=\"./images/g25_20_23_brutal_arch-360ka.png\" alt=\"g25_20_23_brutal_arch-360ka\" width=\"30%\"\u003e\n\u003c/div\u003e\n\u003cdiv\u003e\n   \u003cimg src=\"./images/sand_lines_wgpu-8obz5.png\" alt=\"sand_lines_wgpu-8obz5\" width=\"30%\"\u003e\n   \u003cimg src=\"./images/displacement_2-627iz.png\" alt=\"displacement_2-627iz\" width=\"30%\"\u003e\n   \u003cimg src=\"./images/sand_lines-d8i8g.png\" alt=\"sand_lines-d8i8g.png\" width=\"30%\"\u003e\n\u003c/div\u003e --\u003e\n\nYou can see more screenshots here on github by looking at the auto generated\n[markdown index](index.md) or checkout audio-visual compositions on\n[Instagram][insta].\n\n## Overview\n\nWhether you're curious about generative art, interested in audio-visual\nperformance, or just learning Rust like I am, this project might be worth\nexploring. Originally started as a port of my [p5.js project][p5], it's grown\ninto a surprisingly capable framework that handles the tedious parts of creative\ncoding - like DAW synchronization, hot-reloading configurations, and\nmulti-channel audio processing. While I'm still learning Rust best practices,\nthe project offers some useful features for anyone wanting to experiment with\nalgorithmic art, especially if you're interested in synchronizing visuals with\nmusic. It's set up to work with MIDI controllers and clock, OSC, audio input,\nand even shader programming, making it a fun playground for creative coding\nexperiments.\n\n## Features\n\n- Export images and capture mp4 videos with the press of a button\n- Declarative animation interface with times specified in musical beats, e.g.\n  `1.0` represents 1 beat, `0.5` an eight note, `4.0` a bar, and so on.\n- Sync animations to BPM and frame count, MIDI clock, MIDI Time Code, or OSC\n- Automate parameters with MIDI CC, OSC, CV, or audio with peak, rms, and\n  multiband mechanisms all available through a dead simple API\n- Sync sketch recording with external MIDI Start message which makes it very\n  easy to align your track with the visuals perfectly in post-production\n- Write animations in code or configure your sketch to use an external yaml file\n  that can be hot-reloaded at runtime (similar to live coding - see\n  [Control Scripting](#control-scripting))\n- Declarative per-sketch UI control definitions to easily add sliders, selects,\n  and checkboxes\n- Automatic store/recall of per-sketch UI controls/parameters that can be source\n  controlled\n- Hot reloadable WGSL shaders with various templates to simplify setup\n- **Snapshots** - store and recall all GUI, MIDI, and OSC controls in the UI's\n  Snapshot Editor or by pressing `Shift + Number` to save and\n  `\u003cPlatformModifier\u003e + Number` to recall. Snapshots are interpolated to/from at\n  a configurable musical length from 1/16th note up to 4bars. Great for live\n  performance!\n- **Randomization** - randomize all controls with configurable transition time.\n  Clicking on a slider's label will randomize just that single control, and\n  `\u003cPlatformModifier\u003e + Click` on a label will revert it to its last saved\n  state.\n- **Exclusions** - a column of checkboxes that pops up to the left of each\n  control allowing you to exclude it from **Randomization**, saved with the\n  sketch.\n- Runtime switching of sketches\n- Ability to override sketch BPM via tap tempo to sync with musicians during\n  live performance\n- UI adapts to your operating system's theme preference (see screenshots below)\n\n### Light Mode\n\n![Lattice Controls - Light Theme](assets/ui-light.png)\n\n### Dark Mode\n\n![Lattice Controls - Dark Theme](assets/ui-dark.png)\n\n## Requirements\n\nThis project has been developed on MacOS. I have no idea how it would run on\nother platforms. The project requires or optionally needs:\n\n- Rust\n- Node/NPM or Bun\n- (optional) [just][just] for running commands\n- (optional) ffmpeg available on your path for video exports\n\n## Usage\n\nLattice is still a playground and meant to be cloned and run from source. I do\nplan on making it into a proper app or library eventually. Also my apologies if\nyou plan on cloning you should know the repo contains a few hundred image files\n(using Git LFS) - they are safe to delete. Anyway...\n\nYou will need to run two separate terminal processes: one for the UI controls (A\nTypescript/React app rendered in a [WebView][webview] with [Tao][tao] and\n[Wry][wry], served with [Vite][vite]) and another for the Rust backend.\n\n1. Launch the frontend app server\n\n```sh\ncd ./ui\nbun start # or npm start\n```\n\nIn another terminal window, launch the main Lattice app:\n\n```sh\ncargo run --release -- \u003csketch\u003e\n# or alternatively\njust start \u003csketch\u003e\n```\n\nWhere `sketch` is a file in the src/sketches folder (without the extension) and\nregistered in [src/sketches/mod.rs][module] as well as [src/main.rs][main]. The\nsketch parameter is completely optional. All sketches are selectable in the UI\nand when run without a default sketch, Lattice will load a template sketch.\n\nOptionally you can pass a `timing` positional argument after the required\n`sketch` positional argument to specify what kind of timing system will be used\nto run animations on sketches that support it. Available options include:\n\n#### `frame`\n\nUses Lattice's internal frame system. This is the default and doesn't require\nany external devices to run.\n\n#### `osc`\n\nRequires [assets/L.OscTransport.amxd][osc-transport] to be running in Ableton\nLive. This provides the most reliable syncing mechanism as Ableton does not\nproperly send MIDI SPP messages and doesn't support MTC. See the\n[OSC](#open-sound-control-osc) section for more details.\n\n#### `midi`\n\nUses MIDI clock and MIDI Song Position Pointers (SPP) to stay in sync (e.g. when\na MIDI source loops or you jump to somewhere else in a timeline, your animations\nwill jump or loop accordingly). Bitwig properly sends SPP; Ableton does not.\n\n#### `hybrid`\n\nUses a combination of MIDI clock (for precision) and MIDI Time Code (MTC) to\nstay in sync. This is useful for DAWs that don't support sending SPP but do\nsupport MTC. Ableton, for example, does not support MTC but you can work around\nthat with https://support.showsync.com/sync-tools/livemtc/introduction\n\n### Creating a new sketch:\n\n1. Copy the [template sketch][template] into a new file in the sketches folder.\n2. Rename the `SKETCH_CONFIG.name` field at the top of the file:\n   ```rust\n   pub const SKETCH_CONFIG: SketchConfig = SketchConfig {\n      name: \"template\", // \u003c-- RENAME THIS!\n   ```\n3. Add that filename to the [src/sketches/mod.rs][module]\n4. Add that sketch module to the `register_sketches` call in\n   [src/main.rs][main]:\n5. Run that sketch via command line by `cargo run --release \u003cname\u003e` or\n   `just start \u003cname\u003e` where `name` is what you put in your file's\n   `SKETCH_CONFIG.name` field.\n\n### Audio\n\n#### Multichannel Audio\n\nThe `AudioControls` struct treats each audio channel as an individual control\nsignal with optional slew limiting, suitable for audio-rate or control-rate\nsignals. You can configure the audio device that used in Lattice globally for\nall sketches in the UI via the settings tab. On my computer I'm using the [16\nchannel version of Blackhole][blackhole]. See setup example below:\n\n##### Aggregate Device Setup\n\n![Mac Aggregate Device Setup](assets/aggregate-device-multichannel.png)\n\n\u003e In the above setup I use 1-2 as the main outs and send the multichannel data\n\u003e out to channels 3-18 in my DAW which then appear on Blackhole channels 1-16\n\nSee [audio_controls_dev.rs](src/sketches/dev/audio_controls_dev.rs) or\n[cv_dev.rs](src/sketches/dev/cv_dev.rs) for an example that uses CV.\n\n#### Single Channel, Multiband Audio (_experimental_)\n\nSee [audio_dev.rs](src/sketches/dev/audio_dev.rs) for an example sketch.\n\nThe `Audio` struct in lattice is configured to process the first channel of\nwhatever audio device you have selected in the UI. I am currently doing this via\nAggregate Device on my Mac using [Blackhole 2ch][blackhole] to capture output\nfrom my DAW (setup screenshots below). Note that this module is experimental and\ndoesn't integrate with the rest of Lattice as nicely as `AudioControls` does.\n\n##### Aggregate Device Setup\n\n![Mac Aggregate Device Setup](assets/aggregate-device-setup.png)\n\n##### Routing Audio to Blackhole 2ch `Out(3/4):In(1/2)`\n\n\u003e Note that Blackhole automatically routes whatever its output channels are to\n\u003e its own input, so sending audio out to Blackhole 3/4 will automatically appear\n\u003e on inputs 1/2 in this setup; you don't even need to configure the inputs in\n\u003e Ableton at all for this to work (just as long as you have the output config\n\u003e set to \"Lattice\" and enable the appropriate ouputs in the output config under\n\u003e Live's audio preferences)\n\n![Ableton Live - Blackhole Track Routing](assets/live-blackhole-track-routing.png)\n\n### MIDI\n\nMIDI input and output ports are now global settings in the UI\n\n### MIDI Loopback\n\nTo automate synth parameters in Ableton and Lattice parameters simultaneously\nfrom _the same UI CC control in Live_ (as opposed to a physical control, in\nwhich case you can skip this section), you need to enable MIDI loopback by\nsending MIDI to `Lattice In` and also route `Lattice In` back in to Live to\ncontrol parameters. Here's the routing:\n\n![Live MIDI Preferences](assets/live-midi-prefs.png)\n\nTo use Ableton automation lanes to control Lattice params, follow these steps:\n\n1. Create a MIDI track and clip and add CC automation to it.\n2. In the tracks **MIDI To** router, select `IAC Driver Lattice In` and `Ch. 1`\n\nThose steps are all you need to send MIDI to Lattice to control parameters. As\nfor controlling a live parameter with that same CC, follow these steps:\n\n1. Play your clip containing the CC data\n2. Stop the transport (this is important!)\n3. Enter MIDI Mapping mode.\n4. Locate the parameter to you want to map and select it (make sure it's the\n   last thing you've clicked)\n5. Press the Space bar to start the transport. This should do it!\n\nSee the [midi_test.rs sketch][midi-sketch] for an example of how to map a\ncontrol to something.\n\n\u003e Note: the above instructions are for working without a MIDI controller. When\n\u003e working with a MIDI controller you can just map the MIDI control to an Ableton\n\u003e device knob that can send CC out to Lattice and also map the controller to an\n\u003e Ableton parameter. In this case _you do not_ want Lattice enabled in Ableton's\n\u003e MIDI Input ports at all as that just complicates things.\n\n### Sync Recording\n\nWith MIDI ports configured in your DAW to send clock to Lattice, Lattice is\nalready in a place where you can perfectly sync video recordings with audio from\nyour DAW. Below are steps to setup Ableton Live such that you can record audio\nand video simultaneously when you press Play in the DAW (if you only want to\nrecord video you can just do steps 2 and 4):\n\n1. In Ableton \u003e Preferences \u003e Record, make sure **Start Transport With Record**\n   is set to **Off**\n2. Hit **Q Rec** in Lattice.\n3. Arm tracks in Ableton, arm the transport (Record button)\n4. Now, pressing play in Ableton will also initiate recording in Lattice,\n   likewise pressing Stop in Ableton will stop recording in Lattice.\n\n### Open Sound Control (OSC)\n\nWhile MIDI is great for controlling parameters in the case that a MIDI\ncontroller can send 14bit high resolution MIDI, it sucks otherwise (128 values\njust isn't enough precision for smooth parameter automation). For this reason\nLattice supports OSC and comes with two MaxForLive devices designed to make\nintegration with Ableton Live simpler.\n\n#### L.OscTransport\n\n[assets/L.OscTransport.amxd][osc-transport]\n\n![L.OscTransport MaxForLive Device](assets/osc-transport.png)\n\nPlace this on any track in Ableton and it will send high precision clock and\nexact transport location to Lattice. This should be preferred over using MIDI\nTiming however you should still make sure MIDI ports between Ableton and Lattice\nare configured properly as Lattice still depends on MIDI clock for starting,\nstopping, and syncing video recordings. The default host and port align with\nwhat Lattice expects and can be left alone, though you can configure this in\n[src/config.rs][config].\n\n#### L.OscSend\n\n[assets/L.OscSend.amxd][osc-send]\n\n![L.OscSend MaxForLive Device](assets/osc-send.png)\n\nA super basic OSC value sender. While there are much fancier MaxForLive devices\nthat can send OSC, the \"official\" OSC Send device that comes with Ableton's\nConnection Kit does _not_ send high resolution data, which defeats the entire\npurpose!\n\n### Control Scripting\n\nLattice provides various interfaces for controlling parameters including\n`Controls` for UI (sliders, checkboxes, and selects), `MidiControls` and\n`OscControls` for controlling parameters from an external source,\n`AudioControls` for controlling parameters with audio or CV, and a comprehensive\n`Animation` module that can tween or generate random values and ramp to/from\nthem at musical intervals. While these parameters are simple to setup, it's a\nbit of pain to have to restart the rust sketch every time you want to change an\nanimation or control configuration. For this reason Lattice provides a\n`ControlScript` mechanism that uses yaml for configuration and adds these\ncontrols dynamically and self-updates at runtime when the yaml file is changed.\nYou still have to take care to setup the routings in your sketch (e.g.\n`let radius = model.controls.get(\"radius\")`), but once these routings are in\nplace you are free to edit their ranges, values, timing, etc. See [Control\nScript Test][control-script-test] for a working example or\n[docs/control_script_reference.md](docs/control_script_reference.md) for\ncomprehensive documentation.\n\n## Resources\n\n- https://sotrh.github.io/learn-wgpu\n- https://inconvergent.net/generative/\n- http://www.complexification.net/\n- https://n-e-r-v-o-u-s.com/projects/albums/floraform-system/\n- https://www.andylomas.com/cellularFormImages.html\n- http://www.complexification.net/gallery/machines/sandstroke/\n- https://thebookofshaders.com/\n- https://github.com/jasonwebb/2d-space-colonization-experiments\n- https://paulbourke.net/geometry/\n- https://easings.net/\n\n[p5]: https://github.com/Lokua/p5/tree/main\n[nannou]: https://github.com/nannou-org/nannou\n[insta]: https://www.instagram.com/lokua/\n[just]: https://github.com/casey/just\n[blackhole]: https://existential.audio/blackhole/\n[config]: src/config.rs\n[template]: src/sketches/templates/template.rs\n[midi-sketch]: src/sketches/midi_test.rs\n[module]: src/sketches/mod.rs\n[main]: src/main.rs\n[control-script-test]: src/sketches/scratch/control_script_test.rs\n[osc-transport]: assets/L.OscTransport.amxd\n[osc-send]: assets/L.OscSend.amxd\n[webview]: https://en.wikipedia.org/wiki/WebView\n[tao]: https://github.com/tauri-apps/tao\n[wry]: https://github.com/tauri-apps/wry\n[vite]: https://vite.dev/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flokua%2Flattice","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flokua%2Flattice","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flokua%2Flattice/lists"}