{"id":51462606,"url":"https://github.com/codenlighten/adaptive-ai","last_synced_at":"2026-07-06T07:01:19.719Z","repository":{"id":357600667,"uuid":"1237584073","full_name":"codenlighten/adaptive-ai","owner":"codenlighten","description":null,"archived":false,"fork":false,"pushed_at":"2026-05-13T13:05:26.000Z","size":1529,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-13T14:24:20.420Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/codenlighten.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":"ROADMAP.md","authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-13T10:11:16.000Z","updated_at":"2026-05-13T13:10:21.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/codenlighten/adaptive-ai","commit_stats":null,"previous_names":["codenlighten/adaptive-ai"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/codenlighten/adaptive-ai","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codenlighten%2Fadaptive-ai","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codenlighten%2Fadaptive-ai/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codenlighten%2Fadaptive-ai/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codenlighten%2Fadaptive-ai/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/codenlighten","download_url":"https://codeload.github.com/codenlighten/adaptive-ai/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codenlighten%2Fadaptive-ai/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35180933,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-06T02:00:07.184Z","response_time":106,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-07-06T07:01:18.300Z","updated_at":"2026-07-06T07:01:19.672Z","avatar_url":"https://github.com/codenlighten.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# adaptive-ai\n\n**Delta-sigma weights: multiply-free neural networks with anytime inference.**\n\nA neural-network quantization mechanism that combines the per-cycle simplicity\nof ternary networks with a runtime precision dial. Every cycle of the matmul\nuses only signed additions (no multiplications); the precision–compute tradeoff\nis controlled at *inference time* by truncating a length-T trit stream to\nk ≤ T early samples.\n\nThe result: a single trained model that **adapts its compute per input** —\neasy queries finish in a few cycles, hard queries use the full precision\nbudget. On a heterogeneous workload, a confidence-routed transformer achieves\n**1.9× compute reduction at zero accuracy loss**.\n\n## Headline results\n\n| Task | Pure ternary (BitNet) | fp32 baseline | Delta-Sigma best |\n|---|---:|---:|---:|\n| Damped harmonic oscillator (val MSE) | 1.0e-4 | 2.2e-5 | **6.0e-5** (T=8) |\n| Schrödinger E₀ regression (val MSE)  | 1.6e-3 | 1.1e-4 | **5.2e-4** (T=16) |\n| sklearn digits classification (acc)  | 97.78% | 98.06% | **98.06%** (T=16) — ties fp32 |\n| Char-level transformer LM (val ppl)  | 2.105  | 2.106  | **2.093** (T=8/16) — beats both |\n\n| Anytime inference (T=32 model, single-query) | avg k | compute reduction |\n|---|---:|---:|\n| stop_eps = 0.5 (loose tolerance) | 2.0 | **16×** |\n| stop_eps = 0.1 (medium) | 2.5 | 13× |\n| stop_eps = 0.01 (tight) | 7.9 | 4× |\n\n| Confidence router on mixed workload (70% easy / 25% medium / 5% hard) | avg k of 8 | accuracy | speedup |\n|---|---:|---:|---:|\n| τ = 5.0 | **4.20** | 0.950 | **1.90×** — identical to full-k |\n\n| FPGA (real Yosys + nextpnr place-and-route, iCE40 HX1K) | logic cells | max clock |\n|---|---:|---:|\n| Plain ternary PE (baseline) | 283 (22% of fabric) | 16.83 MHz |\n| **Delta-Sigma PE** | **206 (16%)** | **75.08 MHz** |\n\nSmaller. Faster. Multiply-free. With a runtime precision knob.\n\n## Why this matters\n\nToday's quantization (INT8, INT4, BitNet b1.58) makes precision a\n*training-time* decision. Every input runs at the same precision regardless\nof difficulty.\n\nDelta-sigma weights make precision a **runtime decision per query**. On\nheterogeneous workloads — which describe most real LLM traffic — this means\nspending compute where it matters. The data-center implication is direct:\n\n\u003e At a 70/25/5 easy/medium/hard workload mix, the confidence router yields\n\u003e a ~2× average compute reduction with no accuracy loss. Extrapolated to the\n\u003e ~$1B/year global AI inference electricity bill, that's roughly $500M/year\n\u003e industry-wide if widely adopted.\n\nThese are early-stage numbers, validated on small models. The mechanism is\ndemonstrated end-to-end from PyTorch training through packed-byte storage\nthrough pure-NumPy inference through synthesized Verilog on a real FPGA.\nScaling to billion-parameter LLMs is the next gate.\n\n## Quickstart\n\n```bash\n# Clone + install\ngit clone https://github.com/codenlighten/adaptive-ai\ncd adaptive-ai\n\n# Set up environment\npython3 -m venv venv \u0026\u0026 source venv/bin/activate\npip install -r requirements.txt\n\n# Run unit tests (83 of them)\npython -m pytest tests/\n\n# Train a delta-sigma MLP at T=8 on the damped oscillator\npython -m src.train_dsigma --plot\n\n# Validate across 3 tasks (Schrödinger + digits)\npython -m scripts.validate_dsigma\n\n# Train the delta-sigma transformer\npython -m scripts.train_dsigma_transformer\n\n# Run the anytime-inference demo\npython -m scripts.anytime_inference\n\n# Run the confidence-routed workload simulation\npython -m scripts.workload_simulation\n\n# Run the end-to-end packed-bytes inference demo\npython -m scripts.run_dsigma_packed\n\n# Synthesize the Verilog (requires yowasp-yosys + yowasp-nextpnr-ice40)\nyowasp-yosys -p \"read_verilog hardware/ternary_full_adder.v hardware/dsigma_pe.v; \\\n                 synth_ice40 -top dsigma_pe -json hardware/dsigma_pe.json\"\nyowasp-nextpnr-ice40 --hx1k --json hardware/dsigma_pe.json --asc /tmp/dsigma.asc\n```\n\nThe package exposes a clean public API:\n\n```python\nfrom delta_sigma_nn import DeltaSigmaMLP, DSigmaCharLM, save_dsigma_mlp, dsigma_inference\n\n# Drop-in replacement for nn.Linear, with weights encoded as T-step trit streams\nmodel = DeltaSigmaMLP(in_dim=3, hidden_dim=128, out_dim=1, depth=5, T=8)\n\n# Train as normal\nopt = torch.optim.AdamW(model.parameters(), lr=2e-3)\n# ... standard PyTorch training loop ...\n\n# Anytime inference: choose accuracy/compute tradeoff at runtime\ny, k_used = model.anytime_inference(x, stop_eps=0.01)  # uses ~8 of 8 steps\ny, k_used = model.anytime_inference(x, stop_eps=0.10)  # uses ~3 of 8 steps\n```\n\n## What's in this repo\n\n```\nsrc/                       # core library\n├── delta_sigma.py         # 1st \u0026 2nd-order ΣΔ modulators\n├── dsigma_linear.py       # DeltaSigmaLinear + DeltaSigmaMLP + anytime inference\n├── dsigma_transformer.py  # DSigmaCharLM (causal char-level LM)\n├── dsigma_router.py       # confidence-routed inference (entropy/topk/diff signals)\n├── dsigma_pack.py         # packed-trit serialization + pure-NumPy inference\n├── ternary.py             # BitLinear baseline (BitNet b1.58 style)\n└── ... (45+ source files including quantization variants, EQL, HNN, MoE, etc.)\n\nhardware/\n├── ternary_full_adder.v   # synthesizable balanced-ternary adder\n├── dsigma_pe.v            # delta-sigma processing element\n├── dsigma_systolic.v      # 1-D systolic array of N=8 PEs\n└── synthesize.ys          # Yosys script (results in synthesis_report.md)\n\nscripts/                   # 50+ demos and experiments\n├── train_dsigma.py\n├── validate_dsigma.py\n├── train_dsigma_transformer.py\n├── workload_simulation.py    # confidence router + hyperscale projection\n├── run_dsigma_packed.py      # end-to-end packed inference demo\n├── anytime_inference.py\n└── ...\n\ndelta_sigma_nn/            # pip-installable package\n├── __init__.py\n├── examples/\n│   ├── 01_basic_mlp.py\n│   └── 02_packed_deploy.py\n└── README.md\n\ntests/                     # 83 unit tests\nPAPER.md                   # paper-quality writeup (~4000 words)\nDELTA_SIGMA_WEIGHTS.md     # technical note\n```\n\n## Background\n\nThis repository started as an exploration of balanced-ternary neural networks\nfor physics tasks (BitNet b1.58 applied to oscillator regression, Schrödinger\neigenvalues, Hamiltonian neural networks, equation discovery). The full\nend-to-end chain is included: ternary weights → 5-trits-per-byte packed storage\n→ multiply-free matmul → a simulated Setun-style balanced-ternary ALU →\nsynthesizable Verilog → real FPGA bitstream on a $5 Lattice iCE40.\n\nThe delta-sigma weight mechanism emerged as the most generalizable\ncontribution. The full physics-research codebase remains in the repo\n(`src/hnn.py`, `src/eql.py`, `src/schrodinger.py`, etc.) and is documented in\n`PAPER.md` and `DELTA_SIGMA_WEIGHTS.md`.\n\nNotable physics-side results worth highlighting:\n\n- **Symbolic regression**: a ternary Equation Learner rediscovered the\n  pendulum Hamiltonian exactly (`H = 0.5 p² − cos(q)`) using only 2 of 16\n  candidate basis functions. Sparsity emerges *automatically* from the\n  discrete weight space.\n- **Multi-body Hamiltonians**: the same approach found the exact 8-term\n  Hamiltonian of a 3-mass spring chain.\n- **Cross-task generalization**: a ternary model trained on multiple physics\n  tasks beat fp32 by 5× on out-of-distribution pendulum H regression.\n- **Diffusion model**: ternary 1-D DDPM captured a bimodal double-well\n  distribution without mode collapse.\n\nThese are all reproducible from this repo.\n\n## Paper and writeup\n\n- [`PAPER.md`](PAPER.md) — ~4000-word paper-style writeup suitable for a\n  workshop submission.\n- [`DELTA_SIGMA_WEIGHTS.md`](DELTA_SIGMA_WEIGHTS.md) — technical note\n  focused on the delta-sigma mechanism itself.\n- [`hardware/synthesis_report.md`](hardware/synthesis_report.md) — measured\n  Yosys + nextpnr numbers for the Verilog PE on iCE40.\n\n## Reproducibility\n\nTotal compute to reproduce every experiment is ~2 hours on a modern laptop.\nNo GPU required. No external data downloads (all corpora are generated from\nanalytic physics).\n\n## What's next\n\nTo turn this from \"validated mechanism\" into \"production technique\":\n\n1. **LLM-scale validation** (1B+ parameters on a real corpus). Our largest\n   transformer is 556k params, where DSigma already beats both ternary and\n   fp32 on perplexity. The next gate is whether this scaling holds.\n2. **Vectorized / Triton encode kernel.** The Python encode loop is the\n   bottleneck for the confidence router. A compiled kernel would let the\n   router serve thousands of queries per second instead of dozens.\n3. **Real workload benchmarks.** Run on a public conversational dataset\n   (ShareGPT, OpenAssistant) with standard quality metrics (MMLU, HellaSwag).\n4. **Routing infrastructure prototype.** A real request scheduler that\n   exposes the runtime precision knob to applications, with SLA controls\n   and quality monitoring.\n\n## Citation\n\nIf you build on this work, please cite:\n\n```bibtex\n@misc{ward2026deltasigma,\n  author = {Ward, Gregory J. and Daugherty, Bryan W. and Ryan, Shawn M.},\n  title  = {Delta-Sigma Weights: Multiply-Free Neural Networks with Anytime Inference},\n  year   = {2026},\n  url    = {https://github.com/codenlighten/adaptive-ai},\n  note   = {SmartLedger.Technology}\n}\n```\n\n## Authors\n\n- **Gregory J. Ward** — CTO, [SmartLedger.Technology](https://smartledger.technology) — codenlighten1@gmail.com (corresponding)\n- **Bryan W. Daugherty** — SmartLedger.Technology\n- **Shawn M. Ryan** — SmartLedger.Technology\n\n## License\n\nMIT — see [LICENSE](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodenlighten%2Fadaptive-ai","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcodenlighten%2Fadaptive-ai","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodenlighten%2Fadaptive-ai/lists"}