{"id":50373364,"url":"https://github.com/vardhin/strm","last_synced_at":"2026-05-30T08:04:21.426Z","repository":{"id":339636926,"uuid":"1114765619","full_name":"vardhin/strm","owner":"vardhin","description":null,"archived":false,"fork":false,"pushed_at":"2026-05-23T08:08:16.000Z","size":111918,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-23T10:18:45.253Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vardhin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-11T21:17:42.000Z","updated_at":"2026-05-23T08:08:20.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/vardhin/strm","commit_stats":null,"previous_names":["vardhin/strm"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/vardhin/strm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vardhin%2Fstrm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vardhin%2Fstrm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vardhin%2Fstrm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vardhin%2Fstrm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vardhin","download_url":"https://codeload.github.com/vardhin/strm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vardhin%2Fstrm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33684419,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-05-30T02:00:06.278Z","response_time":92,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-05-30T08:04:21.347Z","updated_at":"2026-05-30T08:04:21.413Z","avatar_url":"https://github.com/vardhin.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# NSSR -- Neuro-Symbolic Recursive Regression\n\nNSSR is a system that **discovers symbolic mathematical expressions from input/output examples**. It combines a tiny recursive transformer (TRM) with a symbolic function registry, letting a neural network *guide* a combinatorial search over function compositions. The system progressively learns higher-level functions by composing primitives (ADD, MUL, etc.) and previously-discovered functions, building a growing vocabulary of reusable symbolic building blocks.\n\n**Core idea**: Instead of training a large neural network to approximate a function, NSSR uses a small recursive transformer to *predict which symbolic compositions are likely correct*, then validates them exactly. The result is a fully interpretable symbolic expression, not a black-box model.\n\n---\n\n## Architecture Overview\n\n```\nInput/Output Examples\n        |\n        v\n  [format_examples]  (search.py)     -- encode as float tensors\n        |\n        v\n  [TRM Model]  (model.py)            -- recursive transformer predicts compositions\n        |\n        v\n  [Guided Search]  (search.py)       -- generate candidates from TRM predictions\n        |                               validate against examples\n        v\n  [Simplify]  (simplify.py)          -- prune redundant steps, strip unused columns\n        |\n        v\n  [Register]  (registry.py + db.py)  -- store as new reusable function\n        |\n        v\n  [Replay Train]  (train.py)         -- retrain TRM on all known functions\n        |\n        v\n  [Next Task...]                      -- vocabulary grows, harder functions become reachable\n```\n\n---\n\n## File-by-File Documentation\n\n### `schema.py` -- Database Schema Definition\n\nDefines the SQLite schema for persisting the symbolic function registry. Single-table design where every function (primitive or learned) lives in one table.\n\n**Constants:**\n\n| Name | Description |\n|---|---|\n| `FUNCTIONS_TABLE` | SQL `CREATE TABLE` statement for the `functions` table. Columns: `id` (primary key), `name` (unique), `arity`, `layer`, `composition` (JSON or NULL for primitives), `constants` (JSON or NULL), `const_mode` (default `'multiplicative'`), `created_at` (timestamp). |\n| `LAYER_INDEX` | SQL `CREATE INDEX` on the `layer` column for efficient layer-based queries. |\n| `ALL_TABLES` | List containing `FUNCTIONS_TABLE`. Used by `db.init_db()` to ensure schema exists. |\n| `ALL_INDEXES` | List containing `LAYER_INDEX`. Used by `db.init_db()` to ensure indexes exist. |\n\n**Design notes:**\n- Primitives have `layer=0` and `composition=NULL`.\n- Learned (composed) functions have `layer\u003e0` and `composition` is a JSON array of `[child_id, [arg_indices]]` pairs.\n- `constants` stores optional fitted constants (e.g., `[0.5]` for KE = 0.5 * m * v^2).\n- `const_mode` is either `\"multiplicative\"` (result *= k) or `\"additive\"` (result += k).\n\n---\n\n### `db.py` -- Database Operations\n\nAll SQLite operations for the symbolic function registry. Every function takes a `sqlite3.Connection` as its first argument. Stateless -- no module-level globals.\n\n#### Functions\n\n| Function | Signature | Description |\n|---|---|---|\n| `init_db` | `(db_path: str = \"checkpoints/symbolic.db\") -\u003e sqlite3.Connection` | Creates or opens the database, ensures the schema (tables + indexes) exists, and returns a connection with `row_factory=sqlite3.Row` enabled. Creates parent directories if needed. |\n| `add_primitive` | `(conn, func_id: int, name: str, arity: int) -\u003e None` | Inserts a primitive function (layer=0, no composition) using `INSERT OR REPLACE`. |\n| `add_learned` | `(conn, func_id: int, name: str, arity: int, composition: list[tuple[int, list[int]]], constants=None, const_mode=\"multiplicative\") -\u003e int` | Inserts a learned function. Auto-calculates `layer` as `max(child layers) + 1`. Serializes `composition` and `constants` to JSON. Returns the computed layer number. Raises `ValueError` if composition is empty. |\n| `get_function` | `(conn, func_id: int) -\u003e dict \\| None` | Returns a single function row as a dict (with parsed JSON fields), or `None` if not found. |\n| `get_composition` | `(conn, func_id: int) -\u003e list[tuple[int, list[int]]]` | Returns the composition steps for a function, or `[]` for primitives. |\n| `get_constants` | `(conn, func_id: int) -\u003e list[float] \\| None` | Returns the constants list for a function, or `None`. |\n| `get_const_mode` | `(conn, func_id: int) -\u003e str` | Returns the `const_mode` for a function, defaulting to `\"multiplicative\"`. |\n| `get_all_functions` | `(conn) -\u003e list[dict]` | Returns every function ordered by layer then id. |\n| `get_functions_by_layer` | `(conn, layer: int) -\u003e list[dict]` | Returns all functions at a specific layer, ordered by id. |\n| `count_functions` | `(conn) -\u003e int` | Total number of registered functions. |\n| `max_layer` | `(conn) -\u003e int` | Highest layer number, or `-1` if the table is empty. |\n| `print_summary` | `(conn) -\u003e None` | Prints a human-readable summary of the entire function database, organized by layer. Shows composition chains for learned functions. |\n| `_row_to_dict` | `(row: sqlite3.Row) -\u003e dict` | **Internal.** Converts a Row to a plain dict, parsing `composition` and `constants` from JSON strings back into Python objects. |\n\n---\n\n### `registry.py` -- Function Registry\n\nThe runtime function registry. Manages all primitives and learned (composed) functions. State is a plain dict passed to every function -- no globals, no classes.\n\n#### Registry State Shape\n\n```python\n{\n    \"functions\":  {int: callable},   # id -\u003e fn(inputs) -\u003e result\n    \"metadata\":   {int: dict},       # id -\u003e {name, arity, layer, ...}\n    \"next_id\":    int,               # next available function id\n    \"loop_id\":    int | None,        # cached id of LOOP primitive\n}\n```\n\n#### Primitives\n\nAll primitives are defined via `_make_primitives()`. Each is `(arity, factory)` where `factory(execute_fn) -\u003e callable(inputs) -\u003e result`. The `execute_fn` indirection lets iteration primitives (LOOP, WHILE, ACCUM) call back into the registry without circular references.\n\n| Name | Arity | Description |\n|---|---|---|\n| `OR` | 2 | Bitwise OR: `inp[0] \\| inp[1]` |\n| `AND` | 2 | Bitwise AND: `inp[0] \u0026 inp[1]` |\n| `NOT` | 1 | Bitwise NOT: `~inp[0]` |\n| `ADD` | 2 | Addition: `inp[0] + inp[1]` |\n| `SUB` | 2 | Subtraction: `inp[0] - inp[1]` |\n| `MUL` | 2 | Multiplication: `inp[0] * inp[1]` |\n| `INC` | 1 | Increment: `inp[0] + 1` |\n| `DEC` | 1 | Decrement: `inp[0] - 1` |\n| `DIV` | 2 | Float division: `inp[0] / inp[1]` (returns 0.0 if divisor is 0) |\n| `LT` | 2 | Less than: `int(inp[0] \u003c inp[1])` |\n| `LTE` | 2 | Less than or equal: `int(inp[0] \u003c= inp[1])` |\n| `GT` | 2 | Greater than: `int(inp[0] \u003e inp[1])` |\n| `GTE` | 2 | Greater than or equal: `int(inp[0] \u003e= inp[1])` |\n| `EQ` | 2 | Equality: `int(inp[0] == inp[1])` |\n| `NEQ` | 2 | Not equal: `int(inp[0] != inp[1])` |\n| `COND` | 3 | Conditional: `inp[1] if inp[0] != 0 else inp[2]` |\n| `CONST` | 1 | Identity/constant: returns `inp[0]` |\n| `NULL` | 1 | Column eraser: always returns 0. Marks an input column as unused during search. |\n| `LOOP` | -1 (variadic) | Iteration. 3-arg form: `body(result)` repeated `count` times from `init`. 4-arg form: `body(result, step_arg)` repeated `count` times. Max 1000 iterations. Enables expressing ADD as `LOOP(INC, b, a)` and MUL as `LOOP(ADD, b, 0, a)`. |\n| `WHILE` | -1 (variadic) | Conditional iteration: `WHILE(cond_fn, body_fn, state, limit)`. Runs `body(state)` while `cond(state) != 0`, up to `limit` iterations (max 1000). |\n| `ACCUM` | -1 (variadic) | Counting iteration: `ACCUM(cond_fn, body_fn, state, counter, limit)`. Like WHILE but increments and returns a counter. |\n\n#### Functions\n\n| Function | Signature | Description |\n|---|---|---|\n| `_make_primitives` | `() -\u003e dict[str, tuple[int, Any]]` | **Internal.** Builds the `{name: (arity, factory)}` dictionary for all built-in primitives. The factory pattern allows iteration primitives to receive an `execute_fn` callback for recursive evaluation. |\n| `_loop_factory` | `(execute_fn) -\u003e callable` | **Internal.** Factory for the LOOP primitive. Supports unary (3-arg) and binary (4-arg) loop bodies. Enforces a 1000-iteration safety limit. |\n| `_while_factory` | `(execute_fn) -\u003e callable` | **Internal.** Factory for the WHILE primitive. Takes `(cond_fn_id, body_fn_id, state, limit)`. Runs body while condition is truthy, up to limit. |\n| `_accum_factory` | `(execute_fn) -\u003e callable` | **Internal.** Factory for the ACCUM primitive. Like WHILE but counts iterations and returns the counter. |\n| `_empty_state` | `() -\u003e dict` | **Internal.** Returns a blank registry state dict. |\n| `execute` | `(state: dict, func_id: int, inputs: list) -\u003e Any` | Executes a registered function by id. Raises `ValueError` if the function id is not found. |\n| `init_registry` | `(conn: sqlite3.Connection) -\u003e dict` | Registers all primitives into both the DB and a fresh state dict. Creates the `_exec` closure so LOOP/WHILE/ACCUM can call back into `execute()`. Returns the populated state. |\n| `load_registry` | `(conn: sqlite3.Connection) -\u003e dict` | Rebuilds the full registry state from an existing database. Re-creates primitive callables from their factories and rebuilds composition closures for learned functions. |\n| `register_learned` | `(conn, state, name, arity, composition, constants=None, const_mode=\"multiplicative\") -\u003e int` | Registers a new composed function. Checks for duplicate names first. Persists to DB via `db.add_learned()`, builds a composition closure, updates state metadata. Returns the new function id. |\n| `_make_composed_fn` | `(state, composition, execute_fn, constants=None, const_mode=\"multiplicative\") -\u003e callable` | **Internal.** Builds a callable that runs a composition step-by-step. The composition is `[(child_func_id, arg_indices), ...]` where arg_indices index into an `available_values` list (starts with inputs, grows as each step appends its result). Special arg index `-1` means literal `0` (used by LOOP for MUL). Applies optional constant scaling/offset at the end. |\n| `vocab_size` | `(state: dict) -\u003e int` | Number of registered functions. Used by the model to size output heads. |\n| `get_name` | `(state, func_id: int) -\u003e str` | Get function name by id. |\n| `get_names` | `(state, func_ids: list[int]) -\u003e list[str]` | Get multiple function names. |\n\n---\n\n### `model.py` -- TRM (Tiny Recursive Reasoning Model)\n\nThe neural network core. Based on the paper **\"Less is More: Recursive Reasoning with Tiny Networks\"** (Jolicoeur-Martineau, 2025).\n\n#### Theory: Why Recursion Instead of Depth?\n\nTraditional transformers gain reasoning power by stacking more layers. TRM takes the opposite approach: use a **tiny** network (2 layers) but **recurse** it many times. This is analogous to how humans solve complex problems -- not by having a bigger brain, but by *thinking longer* about the same problem.\n\nThe key insight is that a small network applied recursively can simulate a much deeper network:\n\n```\nEffective depth = T * (n + 1) * n_layers\n```\n\nWith the default config (T=3, n=6, n_layers=2):\n- Effective depth = 3 * 7 * 2 = **42 effective layers**\n- But only **2 actual layers** of parameters to train\n\nThis is memory-efficient and avoids the vanishing gradient problem of very deep networks.\n\n#### Carry State\n\nThe model maintains two evolving tensor states across recursion steps:\n\n- **y** (answer embedding): The model's current best guess at the answer. This is what the output heads read from.\n- **z** (latent reasoning): Internal \"scratch space\" for intermediate reasoning. Not directly read by output heads.\n\nBoth are `(batch, seq_len, d_model)` tensors, initialized to zeros.\n\n#### Three Levels of Recursion\n\n**1. Latent Recursion** (`_latent_recursion`):\nOne full cycle of reasoning. First, `n` steps refine z (the reasoning state) by feeding it `x + y + z` through the transformer. Then 1 step refines y (the answer) by feeding it `y + z` -- crucially **without x**, so the network learns to distinguish \"reasoning about the input\" from \"updating the answer\".\n\n**2. Deep Recursion** (`_deep_recursion`):\nRuns `T` latent recursion cycles, but only the **last** one computes gradients. The first `T-1` passes are `torch.no_grad()`, meaning they improve y and z \"for free\" (no memory cost for backprop). This gives the effective depth multiplication.\n\n**3. Deep Supervision** (the forward pass):\nThe outer training loop calls `forward()` multiple times (N_sup steps), detaching the carry between each call. This means the model must produce a useful answer at *every* supervision step, not just the final one -- it learns to iteratively refine rather than plan for a single shot.\n\n#### Output Heads\n\nThe model produces 5 prediction heads from pooled y (mean across sequence dimension):\n\n| Head | Output Shape | Description |\n|---|---|---|\n| `primary_logits` | `(batch, n_functions)` | Which function to use as the primary/outer function |\n| `secondary_logits` | `(batch, n_functions)` | Which function to compose with (inner function) |\n| `tertiary_logits` | `(batch, n_functions)` | Combiner function for parallel composition |\n| `composition_logits` | `(batch, 4)` | How to compose: none / sequential / nested / parallel |\n| `halt_logits` | `(batch, 1)` | Whether to stop reasoning (ACT-style halting) |\n\n#### Classes and Dataclasses\n\n| Name | Description |\n|---|---|\n| `Carry` | Dataclass holding `y` (answer embedding), `z` (latent reasoning), `steps` (counter), `halted` (boolean mask). |\n| `_Block` | Single pre-norm transformer block. LayerNorm -\u003e MultiheadAttention (self-attention) -\u003e residual -\u003e LayerNorm -\u003e FFN (Linear -\u003e GELU -\u003e Dropout -\u003e Linear -\u003e Dropout) -\u003e residual. |\n| `TRM` | The full model. Contains input projection, positional embedding, shared transformer blocks, recursion layer norm, and 5 output heads. |\n\n#### TRM Methods\n\n| Method | Signature | Description |\n|---|---|---|\n| `__init__` | `(*, input_dim, seq_len, d_model, n_heads=8, n_layers=2, d_ff=None, dropout=0.1, n_functions, n_recursions=6, T=3)` | Initializes all components. `d_ff` defaults to `d_model * 4`. Creates input projection, learned positional embeddings, `n_layers` transformer blocks, a layer norm for recursion input stabilization, and 5 output heads. |\n| `_apply_blocks` | `(h: Tensor) -\u003e Tensor` | Runs input through all transformer blocks sequentially. |\n| `_latent_recursion` | `(x, y, z) -\u003e (y, z)` | One full latent recursion cycle: n steps of `z = blocks(ln(x+y+z))`, then 1 step of `y = blocks(ln(y+z))`. The layer norm (`ln_recursion`) prevents NaN explosion from repeated additions. |\n| `_deep_recursion` | `(x, y, z) -\u003e (y, z)` | T-1 no-grad passes + 1 grad pass through `_latent_recursion`. Multiplies effective depth by T without proportional memory cost. |\n| `forward` | `(carry: Carry, x: Tensor) -\u003e (Carry, dict[str, Tensor])` | One deep supervision step. Projects input, runs deep recursion, produces output logits from mean-pooled y. Returns new carry (detached) and output dict. |\n\n#### Factory/Helper Functions\n\n| Function | Signature | Description |\n|---|---|---|\n| `create_model` | `(*, input_dim, seq_len, d_model, n_functions, **kwargs) -\u003e TRM` | Factory that creates a TRM with sensible defaults. |\n| `fresh_carry` | `(batch_size, seq_len, d_model) -\u003e Carry` | Creates a zeroed carry state (all tensors are zeros). |\n| `reset_carry` | `(carry: Carry) -\u003e Carry` | Zeros out an existing carry (preserves tensor shapes/device via `zeros_like`). |\n| `resize_heads` | `(model: TRM, old_n: int, new_n: int) -\u003e None` | Expands the 3 function output heads (primary, secondary, tertiary) to accommodate a larger vocabulary. Copies existing weights and initializes new rows with small noise around the existing mean. No-op if `new_n \u003c= old_n`. |\n\n---\n\n### `train.py` -- Training\n\nTraining logic for the TRM model. Uses deep supervision where each training step runs multiple supervision steps with detached carry between them.\n\n#### Constants\n\n| Name | Description |\n|---|---|\n| `COMP_TYPE_INDEX` | Maps composition type strings to integer indices: `{\"none\": 0, \"sequential\": 1, \"nested\": 2, \"parallel\": 3, \"loop_direct\": 0, \"loop_binary\": 0}`. Loop types map to 0 because they are handled separately. |\n\n#### Functions\n\n| Function | Signature | Description |\n|---|---|---|\n| `compute_loss` | `(outputs: dict[str, Tensor], target: dict, batch_size: int) -\u003e Tensor` | Multi-head cross-entropy loss for a single target composition. Computes 5 losses: primary/secondary/tertiary function prediction (cross-entropy), composition type prediction (cross-entropy), and halt prediction (binary cross-entropy). Down-weights secondary loss to 0.1 when `secondary_id` is None, tertiary loss to 0.1 when `tertiary_id` is None. Adds entropy regularization (weight=0.05) on the 3 function heads to prevent the softmax from collapsing to always predict one function. |\n| `train_on_examples` | `(model, optimizer, examples, target, *, input_dim, seq_len, num_epochs=30, n_sup=16) -\u003e list[float]` | Trains the model to predict a specific target composition from examples. Each epoch: creates fresh carry, runs `n_sup` deep supervision steps, accumulates loss at each step with gradient clipping (max norm=1.0). Implements ACT-style early halting -- if the halt head's mean probability exceeds 0.5 after at least 1 step, stops early. Returns per-epoch loss values. |\n| `train_on_replay` | `(model, optimizer, replay_buffer, *, input_dim, seq_len, epochs_per_task=2, n_sup=16) -\u003e list[float]` | Full replay training across all known tasks. Pre-encodes all tasks, then for `n_tasks * epochs_per_task` epochs: iterates through all tasks in sequential order (curriculum-style), trains each with fresh carry and deep supervision. Includes early stopping (patience=5 epochs with no improvement). Returns per-epoch average losses. |\n\n#### Training Theory: Deep Supervision\n\nStandard training would run the model once and compute loss. Deep supervision instead runs the model `N_sup` times, computing and backpropagating loss at each step. The carry state (y, z) persists across steps (detached to prevent gradient flow through the entire chain), forcing the model to learn *iterative refinement* -- each pass should improve the answer.\n\nThis mimics how the model will be used at inference time: the search loop calls `forward()` repeatedly, reading predictions at each step.\n\n---\n\n### `search.py` -- Program Search\n\nTRM-guided combinatorial search over function compositions. The TRM predicts likely candidates, which are then validated exactly against examples.\n\n#### Constants\n\n| Name | Description |\n|---|---|\n| `COMP_TYPES` | `[\"none\", \"sequential\", \"nested\", \"parallel\"]` -- the 4 composition types the model can predict. |\n\n#### Functions\n\n| Function | Signature | Description |\n|---|---|---|\n| `guided` | `(state, model, examples, x_input, *, max_steps=10, max_depth=3, temperature_boost=0.0) -\u003e dict \\| None` | Public entry point. Delegates to `_guided_inner` and returns just the candidate (or None). |\n| `_guided_inner` | `(state, model, examples, x_input, *, max_steps, max_depth, temperature_boost) -\u003e (dict \\| None, float)` | Core search loop. For each step: (1) runs the model forward on the carry, (2) extracts top-k predictions from each head with temperature scaling (starts high=2.0, decays by 0.05/step, boosted on retries), (3) generates candidate compositions from top-k combinations including NULL column variants, (4) validates each candidate against both train and holdout splits, (5) if valid candidates found, returns the simplest one (by `_complexity_score`). If no exact match, tries constant fitting on near misses. Adds small noise to carry between steps for exploration. Uses holdout validation (1/3 split) to prevent overfitting. |\n| `_log_trm_step` | `(state, step, logits, comp_logits, n_functions)` | **Internal.** Logs the TRM's top-5 predictions with probabilities at each search step. Shows what the model is \"thinking\". |\n| `_log_near_misses` | `(state, near_misses, examples, top_n=5)` | **Internal.** Scores and logs the best near-miss candidates by R^2. Helps diagnose why search failed. |\n| `_describe_candidate` | `(state, cand) -\u003e str` | **Internal.** Human-readable description of a candidate (e.g., `\"ADD(MUL(...))\"` or `\"SUB(INC(each))\"`). Includes routing and constant info. |\n| `_null_column_subsets` | `(input_arity, max_null=0) -\u003e list[list[int]]` | **Internal.** Generates all useful subsets of columns to keep (others are NULL'd out). Always includes \"keep all\" and single-column subsets. For 3+ columns, includes all pairs. For 4+ columns, includes all triples. This enables the model to discover that some input columns are irrelevant. |\n| `_find_null_id` | `(state) -\u003e int \\| None` | **Internal.** Finds the NULL primitive's function id by scanning metadata. |\n| `_generate_candidates` | `(state, tops, comp_top, max_depth, input_arity, null_subsets, null_id) -\u003e list[dict]` | **Internal.** Builds candidate dicts from top-k TRM predictions at 3 depth levels. **Depth 1:** single functions with all NULL column/routing variants. **Depth 2:** binary compositions (sequential, nested, parallel) with NULL variants, plus LOOP candidates for unary and binary bodies. **Depth 3:** compositions using all previously-learned functions (not just TRM-predicted ones), expanding the search space for complex compositions. Filters out NULL and LOOP from normal function slots. |\n| `_add_composition_candidates` | `(candidates, state, pid, sid, comp, input_arity, null_subsets)` | **Internal.** Adds sequential/nested candidates with NULL column variants. For sequential: routes kept columns into secondary, primary takes the result. For nested: applies secondary to each kept column, combines with primary. |\n| `_add_parallel_candidates` | `(candidates, state, pid, sid, tid, input_arity, comp, null_subsets, max_routings=30)` | **Internal.** Adds parallel candidates: `tertiary(primary(route1), secondary(route2))`. Generates all routing combinations from kept columns with a cap of 30 to prevent combinatorial explosion. |\n| `_try_fit_constants` | `(state, cand, examples, r2_threshold=0.999) -\u003e dict \\| None` | **Internal.** Tries fitting a single multiplicative or additive constant to a near-miss candidate. First tries `k * produced = expected` (least-squares scale), then `produced + k = expected` (constant offset). Returns augmented candidate if R^2 \u003e 0.999, else None. |\n| `_try_fit_any` | `(state, candidates, examples) -\u003e dict \\| None` | **Internal.** Tries constant fitting on a list of candidates. Returns the first successful fit. |\n| `_fit_scale` | `(produced, expected) -\u003e float \\| None` | **Internal.** Least-squares fit for multiplicative constant: `k = sum(p*e) / sum(p*p)`. Returns None if denominator is near-zero. |\n| `_fit_offset` | `(produced, expected) -\u003e float \\| None` | **Internal.** Fits an additive constant: computes mean of `expected - produced` differences. Returns the offset only if all differences are within 1e-6 of the mean (i.e., it's truly a constant offset, not noise). |\n| `_r2` | `(actual, predicted) -\u003e float` | **Internal.** Computes R-squared (coefficient of determination). Returns 1.0 if both SS_res and SS_tot are 0, `-inf` if SS_tot is 0 but SS_res isn't. |\n| `_candidate` | `(primary_id, secondary_id=None, tertiary_id=None, *, comp_type=\"none\", routing=None, null_columns=None) -\u003e dict` | **Internal.** Helper to construct a candidate dict. |\n| `_complexity_score` | `(c: dict) -\u003e int` | **Internal.** Scores candidate complexity (lower = simpler = preferred). Composition type scores: none=0, sequential/nested=1, parallel=2, loop=3. Adds 1 for each non-None secondary/tertiary id and for constants. |\n| `_cand_key` | `(c: dict) -\u003e tuple` | **Internal.** Creates a hashable key from a candidate for deduplication. Includes all fields: primary, secondary, tertiary, comp_type, routing, null_columns. |\n| `format_examples` | `(examples, *, input_dim, seq_len) -\u003e Tensor` | Encodes examples as float tensors for the TRM. Each input value is encoded as 4 features: `val/100` (normalized), `sign` (+1/-1), `log1p(abs(val))` (magnitude), `fractional part` (decimal component). Output shape: `(batch_size, seq_len, input_dim)`. |\n\n#### Search Theory: NULL-Based Column Elimination\n\nReal-world data often has irrelevant columns. Instead of requiring the user to pre-select columns, the search generates candidates that \"NULL out\" subsets of columns. For a 3-column input `[a, b, c]`, the search tries:\n- Keep all: `[a, b, c]`\n- Keep one: `[a]`, `[b]`, `[c]`\n- Keep pairs: `[a, b]`, `[a, c]`, `[b, c]`\n\nFor each subset, remaining columns are routed into function slots with repetition allowed (so `MUL(a, a)` for squaring is discoverable).\n\n---\n\n### `executor.py` -- Program Executor\n\nRuns composed programs described by candidate dicts against inputs. Validates candidates against input/output examples.\n\n#### Functions\n\n| Function | Signature | Description |\n|---|---|---|\n| `run` | `(state, candidate, inputs) -\u003e Any` | Executes a candidate program on inputs. Calls `_run_base` for the core composition, then applies fitted constants if present (multiplicative or additive). |\n| `_run_base` | `(state, candidate, inputs) -\u003e Any` | **Internal.** Executes the base composition logic. Handles all 6 composition types: **none** (single function with optional routing), **sequential** (`primary(secondary(inputs))`), **nested** (`primary(secondary(x1), secondary(x2), ...)`), **parallel** (`tertiary(primary(route1), secondary(route2))`), **loop_direct** (`LOOP(body, count, init)`), **loop_binary** (`LOOP(body, count, 0, step_arg)`). Respects routing (column selection) when present. |\n| `validate` | `(state, candidate, examples) -\u003e bool` | Checks if a candidate produces the correct output for every example. Uses `math.isclose` with `rel_tol=1e-6, abs_tol=1e-9` for float comparison. Returns False on any exception. |\n| `r_squared` | `(state, func_id, examples) -\u003e float` | Computes R^2 score for a registered function against examples. Used for post-registration verification. Returns `-inf` on any failure. |\n\n---\n\n### `simplify.py` -- Composition Simplifier\n\nAfter search finds a working composition, tries to make it shorter/simpler before registering. Three strategies tried in order.\n\n#### Functions\n\n| Function | Signature | Description |\n|---|---|---|\n| `simplify` | `(state, composition, examples, constants=None, const_mode=\"multiplicative\") -\u003e (composition, effective_arity, used_cols)` | Public entry point. Tries 3 simplification strategies in order. Returns the simplified composition, the effective arity (may be less than input arity if columns were stripped), and `used_cols` (list of original column indices kept, or None if all used). |\n| `_try_single_function` | `(state, examples) -\u003e list[tuple[int, list[int]]] \\| None` | **Strategy 1.** Checks if any existing non-primitive function already solves this exact problem. If so, replaces the entire composition with a single reference. Only used when no constants are involved (constants make the function unique). |\n| `_try_prune` | `(state, composition, examples, constants=None, const_mode=\"multiplicative\") -\u003e list[tuple[int, list[int]]] \\| None` | **Strategy 2.** Tries removing each non-final step from the composition. For each removal, adjusts arg indices that referenced the removed step or later steps. Validates the shortened composition against all examples. Returns the first valid shortened form, or None. |\n| `_strip_unused_inputs` | `(state, composition, examples, constants=None, const_mode=\"multiplicative\") -\u003e (composition, effective_arity, used_cols)` | **Strategy 3.** Identifies input columns that no composition step references, removes them, and remaps arg indices. For example, if a 3-input composition only uses columns 0 and 2, it remaps to a 2-input composition using columns 0 and 1. Validates the remapped composition against remapped examples. |\n| `_validate` | `(state, composition, examples, constants=None, const_mode=\"multiplicative\") -\u003e bool` | **Internal.** Tests a composition against all examples by running it step-by-step: builds an `available_values` list starting with inputs, appends each step's result, checks final result against expected output. Applies constants at the end. |\n| `_complexity` | `(state, composition) -\u003e int` | **Internal.** Scores composition complexity: `n_terms * 100 + max_layer * 10 + total_args`. Lower is better. Heavily penalizes more steps to prefer simpler compositions. |\n\n---\n\n### `main.py` -- Orchestrator / Entry Point\n\nSingle entry point that runs the full NSSR learning pipeline. Coordinates all other modules.\n\n#### Constants\n\n| Name | Description |\n|---|---|\n| `CONFIG` | Global configuration dict: `input_dim=32`, `seq_len=8`, `d_model=128`, `n_heads=8`, `n_layers=2`, `n_recursions=6`, `T=3`, `n_sup=16`, `dropout=0.1`, `lr=1e-4`, `checkpoint_dir=\"checkpoints\"`. |\n| `TARGETS` | List of `(name, examples)` tuples for the progressive learning phase: NAND and XOR, defined using bitwise operations. |\n\n#### Functions\n\n| Function | Signature | Description |\n|---|---|---|\n| `build_composition` | `(candidate, input_arity, loop_id) -\u003e list[tuple[int, list[int]]]` | Converts a search candidate dict into a storable composition list. Handles all 6 composition types. For **none**: direct function call with routing. For **loop_direct**: `[(loop_id, [body_id, 1, 0])]`. For **loop_binary**: `[(loop_id, [body_id, 1, -1, 0])]` where -1 is literal 0. For **sequential**: two steps -- secondary on inputs, primary on result. For **nested**: one step per input column through secondary, then primary combining all. For **parallel**: primary and secondary on their routes, then tertiary combining both results. |\n| `_init_replay_buffer` | `(state)` | **Internal.** Initializes the replay buffer with curriculum tasks if not already present. |\n| `_is_duplicate_discovery` | `(state, candidate) -\u003e bool` | **Internal.** Checks if a candidate is structurally identical to an existing discovery in the replay buffer. Compares all fields: primary_id, secondary_id, tertiary_id, comp_type, routing, constants. |\n| `learn` | `(conn, state, model, optimizer, name, examples, *, max_search_steps=10, max_depth=3, num_epochs=30, max_retries=2) -\u003e (bool, optimizer, float)` | **The main learning pipeline.** 6 phases: **(1)** Initialize replay buffer and pre-train if first call. **(2)** Search with retries (increasing temperature on each retry). **(3)** Check for duplicate discoveries. **(4)** Simplify the found composition (prune, strip columns). **(5)** Register the new function, resize model heads if vocab grew, rebuild optimizer. **(6)** Consolidate all knowledge via full replay training. Returns `(success, optimizer, r2_score)`. The optimizer is returned because it may be recreated if vocab changes. |\n| `curriculum_tasks` | `(state) -\u003e list[dict]` | Generates curriculum tasks: known input/output pairs for each primitive plus composition examples. Each primitive gets a clean task and a noisy variant (with junk columns and routing), teaching the TRM about NULL column elimination. Composition tasks teach sequential (e.g., `INC(MUL(a,b))`), nested (e.g., `ADD(INC(a), INC(b))`), and parallel (e.g., `ADD(INC(a), MUL(b,c))`). Uses fixed random seed (777) for reproducibility. |\n| `save_checkpoint` | `(model, optimizer, state)` | Saves model state_dict, optimizer state_dict, and vocab size to `checkpoints/model.pt`. |\n| `load_checkpoint` | `(model, optimizer, state) -\u003e bool` | Loads a checkpoint. Handles vocab mismatch: if saved model has fewer functions than current registry, creates a temporary model with old vocab size, loads weights, resizes heads, and transfers. Returns False if checkpoint is stale (saved vocab \u003e current vocab) or doesn't exist. |\n| `_fmt_candidate` | `(state, c) -\u003e str` | **Internal.** Formats a candidate as a human-readable string for logging (e.g., `\"ADD(MUL(...))\"`, `\"LOOP(INC, count=b, init=a)\"`). |\n| `main` | `()` | Entry point. Initializes DB + registry, creates model + optimizer, loads checkpoint, runs curriculum pre-training (20 epochs per task), then progressive learning on TARGETS (NAND, XOR). Saves checkpoint and prints DB summary at the end. |\n\n---\n\n### `experiment_noise.py` -- Noise Column Experiment\n\nEnd-to-end experiment testing whether NSSR can ignore irrelevant input columns. Communicates with the system via the HTTP API (requires server.py running).\n\n#### 3-Part Experiment Design\n\n**Part 1 -- Dummy Columns (zero signal):** Tests pure random noise columns. SQUARE(x) = x*x with 2 junk columns. FORCE(m,a) = m*a with interleaved junk columns.\n\n**Part 2 -- Noisy/Correlated Columns:** Tests columns with misleading signal. KE(m,v) = 0.5*m*v^2 with a column correlated to m (multicollinearity trap). PE(m,h) = m*9.81*h with junk columns on both sides.\n\n**Part 3 -- Synthesis with Noise:** TOTAL_E(m,v,h) = KE + PE with junk columns, requiring the model to compose previously-learned functions while still ignoring noise.\n\n#### Functions\n\n| Function | Signature | Description |\n|---|---|---|\n| `api` | `(method, path, json=None) -\u003e dict \\| None` | HTTP helper. Makes a request to the server and returns parsed JSON. Prints error on 4xx/5xx. |\n| `section` | `(title)` | Prints a section header with `=` bars. |\n| `subsection` | `(title)` | Prints a subsection header with `---` bars. |\n| `print_functions` | `(funcs)` | Prints a formatted list of functions with id, name, arity, and layer. |\n| `create_dataset` | `(name, description, examples) -\u003e dict` | Creates a dataset via POST `/datasets`. Prints first 5 examples. |\n| `train_function` | `(target_name, dataset_name, max_depth=5, num_epochs=40, max_search_steps=15) -\u003e dict` | Trains a new function via POST `/train`. Prints result status, R^2, elapsed time, and vocab size. |\n| `evaluate` | `(dataset_name) -\u003e dict` | Evaluates the model on a dataset via POST `/test/eval`. Prints exact match count, best R^2, per-function R^2 scores with ASCII bar charts, and per-example details. |\n| `junk` | `() -\u003e float` | Returns a random float in `[-100, 100]` with 2 decimal places. |\n| `noisy_correlated` | `(val) -\u003e float` | Returns `val + random(-3, 3)` -- a value loosely correlated with the input, creating a multicollinearity trap. |\n| `main` | `()` | Runs the full 3-part experiment: preflight check, dummy columns (SQUARE, FORCE), noisy columns (KE, PE), synthesis (TOTAL_E). Prints a final summary with all function compositions and training history. |\n\n---\n\n## Interaction Diagram (Mermaid)\n\n```mermaid\ngraph TB\n    subgraph schema.py [\"schema.py -- Database Schema\"]\n        SCHEMA_FUNCTIONS_TABLE[\"FUNCTIONS_TABLE\u003cbr/\u003e(SQL CREATE TABLE)\"]\n        SCHEMA_LAYER_INDEX[\"LAYER_INDEX\u003cbr/\u003e(SQL CREATE INDEX)\"]\n        SCHEMA_ALL_TABLES[\"ALL_TABLES\"]\n        SCHEMA_ALL_INDEXES[\"ALL_INDEXES\"]\n    end\n\n    subgraph db.py [\"db.py -- Database Operations\"]\n        DB_init_db[\"init_db()\u003cbr/\u003eCreate/open DB, apply schema\"]\n        DB_add_primitive[\"add_primitive()\u003cbr/\u003eInsert primitive function\"]\n        DB_add_learned[\"add_learned()\u003cbr/\u003eInsert learned function,\u003cbr/\u003eauto-calc layer\"]\n        DB_get_function[\"get_function()\u003cbr/\u003eGet single function row\"]\n        DB_get_composition[\"get_composition()\u003cbr/\u003eGet composition steps\"]\n        DB_get_constants[\"get_constants()\u003cbr/\u003eGet constants list\"]\n        DB_get_const_mode[\"get_const_mode()\u003cbr/\u003eGet const_mode string\"]\n        DB_get_all_functions[\"get_all_functions()\u003cbr/\u003eAll functions by layer\"]\n        DB_get_functions_by_layer[\"get_functions_by_layer()\u003cbr/\u003eFunctions at specific layer\"]\n        DB_count_functions[\"count_functions()\u003cbr/\u003eTotal function count\"]\n        DB_max_layer[\"max_layer()\u003cbr/\u003eHighest layer number\"]\n        DB_print_summary[\"print_summary()\u003cbr/\u003eHuman-readable DB dump\"]\n        DB__row_to_dict[\"_row_to_dict()\u003cbr/\u003eRow -\u003e dict with JSON parse\"]\n    end\n\n    subgraph registry.py [\"registry.py -- Function Registry\"]\n        REG_PRIMITIVES[\"PRIMITIVES\u003cbr/\u003e(module-level dict)\"]\n        REG__make_primitives[\"_make_primitives()\u003cbr/\u003eBuild all primitive defs\"]\n        REG__loop_factory[\"_loop_factory()\u003cbr/\u003eLOOP primitive factory\"]\n        REG__while_factory[\"_while_factory()\u003cbr/\u003eWHILE primitive factory\"]\n        REG__accum_factory[\"_accum_factory()\u003cbr/\u003eACCUM primitive factory\"]\n        REG__empty_state[\"_empty_state()\u003cbr/\u003eBlank registry state\"]\n        REG_execute[\"execute()\u003cbr/\u003eRun function by id\"]\n        REG_init_registry[\"init_registry()\u003cbr/\u003eRegister primitives,\u003cbr/\u003ereturn state\"]\n        REG_load_registry[\"load_registry()\u003cbr/\u003eRebuild state from DB\"]\n        REG_register_learned[\"register_learned()\u003cbr/\u003eRegister new composed fn\"]\n        REG__make_composed_fn[\"_make_composed_fn()\u003cbr/\u003eBuild composition closure\"]\n        REG_vocab_size[\"vocab_size()\u003cbr/\u003eCount of functions\"]\n        REG_get_name[\"get_name()\u003cbr/\u003eFunction name by id\"]\n        REG_get_names[\"get_names()\u003cbr/\u003eMultiple names by ids\"]\n    end\n\n    subgraph model.py [\"model.py -- TRM Neural Network\"]\n        MOD_Carry[\"Carry (dataclass)\u003cbr/\u003ey, z, steps, halted\"]\n        MOD__Block[\"_Block (nn.Module)\u003cbr/\u003ePre-norm transformer block\u003cbr/\u003eLN -\u003e MHA -\u003e Residual -\u003e LN -\u003e FFN -\u003e Residual\"]\n        MOD_TRM[\"TRM (nn.Module)\u003cbr/\u003eTiny Recursive Model\"]\n        MOD_TRM_init[\"TRM.__init__()\u003cbr/\u003einput_proj, pos_emb,\u003cbr/\u003eblocks, heads\"]\n        MOD_TRM_apply_blocks[\"TRM._apply_blocks()\u003cbr/\u003eRun all blocks\"]\n        MOD_TRM_latent_recursion[\"TRM._latent_recursion()\u003cbr/\u003en steps: z=net(x+y+z)\u003cbr/\u003e1 step: y=net(y+z)\"]\n        MOD_TRM_deep_recursion[\"TRM._deep_recursion()\u003cbr/\u003eT-1 no_grad + 1 grad pass\"]\n        MOD_TRM_forward[\"TRM.forward()\u003cbr/\u003eOne supervision step:\u003cbr/\u003eembed -\u003e deep_recurse -\u003e heads\"]\n        MOD_create_model[\"create_model()\u003cbr/\u003eFactory with defaults\"]\n        MOD_fresh_carry[\"fresh_carry()\u003cbr/\u003eZeroed carry state\"]\n        MOD_reset_carry[\"reset_carry()\u003cbr/\u003eZero existing carry\"]\n        MOD_resize_heads[\"resize_heads()\u003cbr/\u003eExpand output heads\u003cbr/\u003efor new vocab\"]\n    end\n\n    subgraph train.py [\"train.py -- Training\"]\n        TRN_COMP_TYPE_INDEX[\"COMP_TYPE_INDEX\u003cbr/\u003e(string -\u003e int map)\"]\n        TRN_compute_loss[\"compute_loss()\u003cbr/\u003eMulti-head CE + entropy reg\"]\n        TRN_train_on_examples[\"train_on_examples()\u003cbr/\u003eTrain on single task\u003cbr/\u003ewith deep supervision\"]\n        TRN_train_on_replay[\"train_on_replay()\u003cbr/\u003eFull replay: all tasks,\u003cbr/\u003eearly stopping\"]\n    end\n\n    subgraph search.py [\"search.py -- Program Search\"]\n        SCH_COMP_TYPES[\"COMP_TYPES\u003cbr/\u003e[none, seq, nested, parallel]\"]\n        SCH_guided[\"guided()\u003cbr/\u003ePublic search entry\"]\n        SCH__guided_inner[\"_guided_inner()\u003cbr/\u003eCore search loop:\u003cbr/\u003eTRM predict -\u003e generate -\u003e validate\"]\n        SCH__log_trm_step[\"_log_trm_step()\u003cbr/\u003eLog TRM predictions\"]\n        SCH__log_near_misses[\"_log_near_misses()\u003cbr/\u003eScore \u0026 log best misses\"]\n        SCH__describe_candidate[\"_describe_candidate()\u003cbr/\u003eHuman-readable candidate\"]\n        SCH__null_column_subsets[\"_null_column_subsets()\u003cbr/\u003eGenerate kept-column subsets\"]\n        SCH__find_null_id[\"_find_null_id()\u003cbr/\u003eFind NULL primitive id\"]\n        SCH__generate_candidates[\"_generate_candidates()\u003cbr/\u003eBuild candidates from\u003cbr/\u003etop-k predictions (3 depths)\"]\n        SCH__add_composition_candidates[\"_add_composition_candidates()\u003cbr/\u003eSeq/nested + NULL variants\"]\n        SCH__add_parallel_candidates[\"_add_parallel_candidates()\u003cbr/\u003eParallel + routing combos\"]\n        SCH__try_fit_constants[\"_try_fit_constants()\u003cbr/\u003eFit scale or offset\"]\n        SCH__try_fit_any[\"_try_fit_any()\u003cbr/\u003eTry fitting on candidates\"]\n        SCH__fit_scale[\"_fit_scale()\u003cbr/\u003eLeast-squares scale\"]\n        SCH__fit_offset[\"_fit_offset()\u003cbr/\u003eConstant offset\"]\n        SCH__r2[\"_r2()\u003cbr/\u003eR-squared metric\"]\n        SCH__candidate[\"_candidate()\u003cbr/\u003eConstruct candidate dict\"]\n        SCH__complexity_score[\"_complexity_score()\u003cbr/\u003eSimplicity preference\"]\n        SCH__cand_key[\"_cand_key()\u003cbr/\u003eHashable dedup key\"]\n        SCH_format_examples[\"format_examples()\u003cbr/\u003eEncode examples as tensors\"]\n    end\n\n    subgraph executor.py [\"executor.py -- Program Executor\"]\n        EXE_run[\"run()\u003cbr/\u003eExecute candidate + constants\"]\n        EXE__run_base[\"_run_base()\u003cbr/\u003eCore composition execution\u003cbr/\u003e(6 comp types)\"]\n        EXE_validate[\"validate()\u003cbr/\u003eCheck candidate vs examples\"]\n        EXE_r_squared[\"r_squared()\u003cbr/\u003eR² for registered function\"]\n    end\n\n    subgraph simplify.py [\"simplify.py -- Composition Simplifier\"]\n        SIM_simplify[\"simplify()\u003cbr/\u003ePublic: try 3 strategies\"]\n        SIM__try_single_function[\"_try_single_function()\u003cbr/\u003eStrategy 1: existing fn match\"]\n        SIM__try_prune[\"_try_prune()\u003cbr/\u003eStrategy 2: remove steps\"]\n        SIM__strip_unused_inputs[\"_strip_unused_inputs()\u003cbr/\u003eStrategy 3: drop unused cols\"]\n        SIM__validate[\"_validate()\u003cbr/\u003eTest composition step-by-step\"]\n        SIM__complexity[\"_complexity()\u003cbr/\u003eComplexity score\"]\n    end\n\n    subgraph main.py [\"main.py -- Orchestrator\"]\n        MAIN_CONFIG[\"CONFIG\u003cbr/\u003e(global hyperparams)\"]\n        MAIN_TARGETS[\"TARGETS\u003cbr/\u003e[NAND, XOR]\"]\n        MAIN_build_composition[\"build_composition()\u003cbr/\u003eCandidate -\u003e storable comp\"]\n        MAIN__init_replay_buffer[\"_init_replay_buffer()\u003cbr/\u003eInit buffer with curriculum\"]\n        MAIN__is_duplicate_discovery[\"_is_duplicate_discovery()\u003cbr/\u003eCheck structural duplicate\"]\n        MAIN_learn[\"learn()\u003cbr/\u003eFull pipeline: search -\u003e\u003cbr/\u003esimplify -\u003e register -\u003e train\"]\n        MAIN_curriculum_tasks[\"curriculum_tasks()\u003cbr/\u003eGenerate primitive + comp\u003cbr/\u003ecurriculum with noise variants\"]\n        MAIN_save_checkpoint[\"save_checkpoint()\u003cbr/\u003eSave model + optimizer\"]\n        MAIN_load_checkpoint[\"load_checkpoint()\u003cbr/\u003eLoad + handle vocab mismatch\"]\n        MAIN__fmt_candidate[\"_fmt_candidate()\u003cbr/\u003eFormat candidate for logging\"]\n        MAIN_main[\"main()\u003cbr/\u003eEntry point: init -\u003e pretrain\u003cbr/\u003e-\u003e learn NAND, XOR -\u003e save\"]\n    end\n\n    subgraph experiment_noise.py [\"experiment_noise.py -- Noise Experiment\"]\n        EXP_api[\"api()\u003cbr/\u003eHTTP request helper\"]\n        EXP_section[\"section()\u003cbr/\u003ePrint section header\"]\n        EXP_subsection[\"subsection()\u003cbr/\u003ePrint subsection header\"]\n        EXP_print_functions[\"print_functions()\u003cbr/\u003eFormat function list\"]\n        EXP_create_dataset[\"create_dataset()\u003cbr/\u003ePOST /datasets\"]\n        EXP_train_function[\"train_function()\u003cbr/\u003ePOST /train\"]\n        EXP_evaluate[\"evaluate()\u003cbr/\u003ePOST /test/eval\"]\n        EXP_junk[\"junk()\u003cbr/\u003eRandom noise value\"]\n        EXP_noisy_correlated[\"noisy_correlated()\u003cbr/\u003eCorrelated noise value\"]\n        EXP_main[\"main()\u003cbr/\u003eRun 3-part experiment\"]\n    end\n\n    %% === CROSS-FILE DEPENDENCIES ===\n\n    %% schema.py -\u003e db.py\n    SCHEMA_ALL_TABLES --\u003e|\"imported\"| DB_init_db\n    SCHEMA_ALL_INDEXES --\u003e|\"imported\"| DB_init_db\n\n    %% db.py -\u003e registry.py\n    DB_add_primitive --\u003e|\"called by\"| REG_init_registry\n    DB_add_learned --\u003e|\"called by\"| REG_register_learned\n    DB_get_all_functions --\u003e|\"called by\"| REG_load_registry\n    DB_get_composition --\u003e|\"called by\"| REG_load_registry\n    DB_get_constants --\u003e|\"called by\"| REG_load_registry\n    DB_get_const_mode --\u003e|\"called by\"| REG_load_registry\n\n    %% db.py internal\n    DB_get_function --\u003e|\"called by\"| DB_get_composition\n    DB_get_function --\u003e|\"called by\"| DB_get_constants\n    DB_get_function --\u003e|\"called by\"| DB_get_const_mode\n    DB_get_function --\u003e|\"called by\"| DB_print_summary\n    DB__row_to_dict --\u003e|\"called by\"| DB_get_function\n    DB__row_to_dict --\u003e|\"called by\"| DB_get_all_functions\n    DB__row_to_dict --\u003e|\"called by\"| DB_get_functions_by_layer\n    DB_max_layer --\u003e|\"called by\"| DB_print_summary\n    DB_get_functions_by_layer --\u003e|\"called by\"| DB_print_summary\n\n    %% registry.py internal\n    REG__make_primitives --\u003e|\"builds\"| REG_PRIMITIVES\n    REG__loop_factory --\u003e|\"used by\"| REG__make_primitives\n    REG__while_factory --\u003e|\"used by\"| REG__make_primitives\n    REG__accum_factory --\u003e|\"used by\"| REG__make_primitives\n    REG__empty_state --\u003e|\"called by\"| REG_init_registry\n    REG__empty_state --\u003e|\"called by\"| REG_load_registry\n    REG__make_composed_fn --\u003e|\"called by\"| REG_load_registry\n    REG__make_composed_fn --\u003e|\"called by\"| REG_register_learned\n    REG_execute --\u003e|\"callback into\"| REG__loop_factory\n    REG_execute --\u003e|\"callback into\"| REG__while_factory\n    REG_execute --\u003e|\"callback into\"| REG__accum_factory\n    REG_execute --\u003e|\"callback into\"| REG__make_composed_fn\n\n    %% model.py internal\n    MOD__Block --\u003e|\"used by\"| MOD_TRM_init\n    MOD_TRM_apply_blocks --\u003e|\"calls\"| MOD__Block\n    MOD_TRM_latent_recursion --\u003e|\"calls\"| MOD_TRM_apply_blocks\n    MOD_TRM_deep_recursion --\u003e|\"calls\"| MOD_TRM_latent_recursion\n    MOD_TRM_forward --\u003e|\"calls\"| MOD_TRM_deep_recursion\n    MOD_TRM_forward --\u003e|\"reads\"| MOD_Carry\n    MOD_create_model --\u003e|\"creates\"| MOD_TRM\n    MOD_fresh_carry --\u003e|\"creates\"| MOD_Carry\n\n    %% registry.py -\u003e executor.py\n    REG_execute --\u003e|\"called by\"| EXE__run_base\n    REG_execute --\u003e|\"called by\"| EXE_r_squared\n\n    %% executor.py internal\n    EXE__run_base --\u003e|\"called by\"| EXE_run\n    EXE_run --\u003e|\"called by\"| EXE_validate\n\n    %% model.py -\u003e train.py\n    MOD_fresh_carry --\u003e|\"called by\"| TRN_train_on_examples\n    MOD_fresh_carry --\u003e|\"called by\"| TRN_train_on_replay\n    MOD_TRM_forward --\u003e|\"called by\"| TRN_train_on_examples\n    MOD_TRM_forward --\u003e|\"called by\"| TRN_train_on_replay\n\n    %% search.py -\u003e executor.py\n    EXE_validate --\u003e|\"called by\"| SCH__guided_inner\n    EXE_run --\u003e|\"called by\"| SCH__try_fit_constants\n    EXE_run --\u003e|\"called by\"| SCH__log_near_misses\n    EXE_run --\u003e|\"called by\"| SCH__guided_inner\n\n    %% search.py -\u003e model.py\n    MOD_fresh_carry --\u003e|\"called by\"| SCH__guided_inner\n    MOD_TRM_forward --\u003e|\"called by\"| SCH__guided_inner\n\n    %% search.py -\u003e registry.py\n    REG_vocab_size --\u003e|\"called by\"| SCH__guided_inner\n\n    %% search.py -\u003e train.py\n    SCH_format_examples --\u003e|\"called by\"| TRN_train_on_examples\n\n    %% search.py internal\n    SCH__guided_inner --\u003e|\"called by\"| SCH_guided\n    SCH__generate_candidates --\u003e|\"called by\"| SCH__guided_inner\n    SCH__null_column_subsets --\u003e|\"called by\"| SCH__guided_inner\n    SCH__find_null_id --\u003e|\"called by\"| SCH__guided_inner\n    SCH__log_trm_step --\u003e|\"called by\"| SCH__guided_inner\n    SCH__log_near_misses --\u003e|\"called by\"| SCH__guided_inner\n    SCH__try_fit_any --\u003e|\"called by\"| SCH__guided_inner\n    SCH__cand_key --\u003e|\"called by\"| SCH__guided_inner\n    SCH__complexity_score --\u003e|\"called by\"| SCH__guided_inner\n    SCH__add_composition_candidates --\u003e|\"called by\"| SCH__generate_candidates\n    SCH__add_parallel_candidates --\u003e|\"called by\"| SCH__generate_candidates\n    SCH__candidate --\u003e|\"called by\"| SCH__generate_candidates\n    SCH__candidate --\u003e|\"called by\"| SCH__add_composition_candidates\n    SCH__candidate --\u003e|\"called by\"| SCH__add_parallel_candidates\n    SCH__try_fit_constants --\u003e|\"called by\"| SCH__try_fit_any\n    SCH__fit_scale --\u003e|\"called by\"| SCH__try_fit_constants\n    SCH__fit_offset --\u003e|\"called by\"| SCH__try_fit_constants\n    SCH__r2 --\u003e|\"called by\"| SCH__try_fit_constants\n    SCH__r2 --\u003e|\"called by\"| SCH__guided_inner\n    SCH__r2 --\u003e|\"called by\"| SCH__log_near_misses\n    SCH__describe_candidate --\u003e|\"called by\"| SCH__log_near_misses\n\n    %% simplify.py -\u003e registry.py\n    REG_execute --\u003e|\"called by\"| SIM__try_single_function\n    REG_execute --\u003e|\"called by\"| SIM__validate\n\n    %% simplify.py internal\n    SIM__try_single_function --\u003e|\"called by\"| SIM_simplify\n    SIM__try_prune --\u003e|\"called by\"| SIM_simplify\n    SIM__strip_unused_inputs --\u003e|\"called by\"| SIM_simplify\n    SIM__validate --\u003e|\"called by\"| SIM__try_prune\n    SIM__validate --\u003e|\"called by\"| SIM__strip_unused_inputs\n    SIM__complexity --\u003e|\"called by\"| SIM_simplify\n\n    %% main.py -\u003e all modules\n    DB_init_db --\u003e|\"called by\"| MAIN_main\n    DB_print_summary --\u003e|\"called by\"| MAIN_main\n    REG_init_registry --\u003e|\"called by\"| MAIN_main\n    REG_register_learned --\u003e|\"called by\"| MAIN_learn\n    REG_vocab_size --\u003e|\"called by\"| MAIN_main\n    REG_vocab_size --\u003e|\"called by\"| MAIN_learn\n    REG_get_name --\u003e|\"called by\"| MAIN__fmt_candidate\n    REG_get_names --\u003e|\"called by\"| MAIN_main\n    MOD_create_model --\u003e|\"called by\"| MAIN_main\n    MOD_create_model --\u003e|\"called by\"| MAIN_load_checkpoint\n    MOD_resize_heads --\u003e|\"called by\"| MAIN_learn\n    MOD_resize_heads --\u003e|\"called by\"| MAIN_load_checkpoint\n    MOD_fresh_carry --\u003e|\"called by\"| MAIN_learn\n    SCH_guided --\u003e|\"called by\"| MAIN_learn\n    SCH_format_examples --\u003e|\"called by\"| MAIN_learn\n    TRN_train_on_examples --\u003e|\"called by\"| MAIN_main\n    TRN_train_on_replay --\u003e|\"called by\"| MAIN_learn\n    SIM_simplify --\u003e|\"called by\"| MAIN_learn\n    EXE_r_squared --\u003e|\"called by\"| MAIN_learn\n\n    %% main.py internal\n    MAIN_build_composition --\u003e|\"called by\"| MAIN_learn\n    MAIN__init_replay_buffer --\u003e|\"called by\"| MAIN_learn\n    MAIN__is_duplicate_discovery --\u003e|\"called by\"| MAIN_learn\n    MAIN_curriculum_tasks --\u003e|\"called by\"| MAIN__init_replay_buffer\n    MAIN_curriculum_tasks --\u003e|\"called by\"| MAIN_main\n    MAIN_save_checkpoint --\u003e|\"called by\"| MAIN_main\n    MAIN_load_checkpoint --\u003e|\"called by\"| MAIN_main\n    MAIN__fmt_candidate --\u003e|\"called by\"| MAIN_learn\n    MAIN_learn --\u003e|\"called by\"| MAIN_main\n\n    %% experiment_noise.py internal\n    EXP_api --\u003e|\"called by\"| EXP_create_dataset\n    EXP_api --\u003e|\"called by\"| EXP_train_function\n    EXP_api --\u003e|\"called by\"| EXP_evaluate\n    EXP_api --\u003e|\"called by\"| EXP_main\n    EXP_section --\u003e|\"called by\"| EXP_main\n    EXP_subsection --\u003e|\"called by\"| EXP_main\n    EXP_print_functions --\u003e|\"called by\"| EXP_main\n    EXP_create_dataset --\u003e|\"called by\"| EXP_main\n    EXP_train_function --\u003e|\"called by\"| EXP_main\n    EXP_evaluate --\u003e|\"called by\"| EXP_main\n    EXP_junk --\u003e|\"called by\"| EXP_main\n    EXP_noisy_correlated --\u003e|\"called by\"| EXP_main\n\n    %% experiment_noise.py -\u003e server (external)\n    EXP_api -.-\u003e|\"HTTP to server.py\"| SERVER[\"server.py\u003cbr/\u003e(not documented)\"]\n\n    %% Styling\n    classDef schemaStyle fill:#e8f5e9,stroke:#388e3c\n    classDef dbStyle fill:#e3f2fd,stroke:#1976d2\n    classDef regStyle fill:#fff3e0,stroke:#f57c00\n    classDef modelStyle fill:#fce4ec,stroke:#c62828\n    classDef trainStyle fill:#f3e5f5,stroke:#7b1fa2\n    classDef searchStyle fill:#e0f7fa,stroke:#0097a7\n    classDef execStyle fill:#fff8e1,stroke:#f9a825\n    classDef simpStyle fill:#f1f8e9,stroke:#689f38\n    classDef mainStyle fill:#e8eaf6,stroke:#3f51b5\n    classDef expStyle fill:#fafafa,stroke:#616161\n\n    class SCHEMA_FUNCTIONS_TABLE,SCHEMA_LAYER_INDEX,SCHEMA_ALL_TABLES,SCHEMA_ALL_INDEXES schemaStyle\n    class DB_init_db,DB_add_primitive,DB_add_learned,DB_get_function,DB_get_composition,DB_get_constants,DB_get_const_mode,DB_get_all_functions,DB_get_functions_by_layer,DB_count_functions,DB_max_layer,DB_print_summary,DB__row_to_dict dbStyle\n    class REG_PRIMITIVES,REG__make_primitives,REG__loop_factory,REG__while_factory,REG__accum_factory,REG__empty_state,REG_execute,REG_init_registry,REG_load_registry,REG_register_learned,REG__make_composed_fn,REG_vocab_size,REG_get_name,REG_get_names regStyle\n    class MOD_Carry,MOD__Block,MOD_TRM,MOD_TRM_init,MOD_TRM_apply_blocks,MOD_TRM_latent_recursion,MOD_TRM_deep_recursion,MOD_TRM_forward,MOD_create_model,MOD_fresh_carry,MOD_reset_carry,MOD_resize_heads modelStyle\n    class TRN_COMP_TYPE_INDEX,TRN_compute_loss,TRN_train_on_examples,TRN_train_on_replay trainStyle\n    class SCH_COMP_TYPES,SCH_guided,SCH__guided_inner,SCH__log_trm_step,SCH__log_near_misses,SCH__describe_candidate,SCH__null_column_subsets,SCH__find_null_id,SCH__generate_candidates,SCH__add_composition_candidates,SCH__add_parallel_candidates,SCH__try_fit_constants,SCH__try_fit_any,SCH__fit_scale,SCH__fit_offset,SCH__r2,SCH__candidate,SCH__complexity_score,SCH__cand_key,SCH_format_examples searchStyle\n    class EXE_run,EXE__run_base,EXE_validate,EXE_r_squared execStyle\n    class SIM_simplify,SIM__try_single_function,SIM__try_prune,SIM__strip_unused_inputs,SIM__validate,SIM__complexity simpStyle\n    class MAIN_CONFIG,MAIN_TARGETS,MAIN_build_composition,MAIN__init_replay_buffer,MAIN__is_duplicate_discovery,MAIN_learn,MAIN_curriculum_tasks,MAIN_save_checkpoint,MAIN_load_checkpoint,MAIN__fmt_candidate,MAIN_main mainStyle\n    class EXP_api,EXP_section,EXP_subsection,EXP_print_functions,EXP_create_dataset,EXP_train_function,EXP_evaluate,EXP_junk,EXP_noisy_correlated,EXP_main expStyle\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvardhin%2Fstrm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvardhin%2Fstrm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvardhin%2Fstrm/lists"}