{"id":50874509,"url":"https://github.com/sbryngelson/aneforge","last_synced_at":"2026-06-19T12:00:39.935Z","repository":{"id":364393829,"uuid":"1267734215","full_name":"sbryngelson/ANEForge","owner":"sbryngelson","description":"Pythonic binding to the Apple Neural Engine","archived":false,"fork":false,"pushed_at":"2026-06-17T22:32:31.000Z","size":6306,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-18T11:33:20.947Z","etag":null,"topics":["accelerated-computing","apple-silicon","machine-learning","neural-networks"],"latest_commit_sha":null,"homepage":"https://aneforge.readthedocs.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sbryngelson.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":"docs/roadmap.md","authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-12T20:15:28.000Z","updated_at":"2026-06-17T22:32:35.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/sbryngelson/ANEForge","commit_stats":null,"previous_names":["sbryngelson/aneforge"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/sbryngelson/ANEForge","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sbryngelson%2FANEForge","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sbryngelson%2FANEForge/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sbryngelson%2FANEForge/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sbryngelson%2FANEForge/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sbryngelson","download_url":"https://codeload.github.com/sbryngelson/ANEForge/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sbryngelson%2FANEForge/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34530302,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-19T02:00:06.005Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["accelerated-computing","apple-silicon","machine-learning","neural-networks"],"created_at":"2026-06-15T08:30:39.668Z","updated_at":"2026-06-19T12:00:39.870Z","avatar_url":"https://github.com/sbryngelson.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ANEForge\n\n[![CI](https://github.com/sbryngelson/ANEForge/actions/workflows/ci.yml/badge.svg)](https://github.com/sbryngelson/ANEForge/actions/workflows/ci.yml)\n[![PyPI](https://img.shields.io/pypi/v/aneforge?color=2e6b69)](https://pypi.org/project/aneforge/)\n[![Docs](https://readthedocs.org/projects/aneforge/badge/?version=latest)](https://aneforge.readthedocs.io)\n[![arXiv](https://img.shields.io/badge/arXiv-2606.17090-b31b1b.svg)](https://arxiv.org/abs/2606.17090)\n[![Zenodo](https://zenodo.org/badge/DOI/10.5281/zenodo.20672609.svg)](https://doi.org/10.5281/zenodo.20672609)\n[![License: MIT](https://img.shields.io/badge/License-MIT-2e6b69.svg)](LICENSE)\n[![Apple Silicon](https://img.shields.io/badge/Apple%20Silicon-macOS%2014%2B-b84310.svg)](#install)\n\n**Train and run neural networks directly on the Apple Neural Engine, from\nPython, with no CoreML.**\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/assets/demo.png\" width=\"680\"\n       alt=\"A small transformer trains from scratch and generates text live on the Apple Neural Engine\"\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003csub\u003eA transformer training from scratch on the engine (forward, backward, and\n  Adam), then completing a prompt. Reproduce with \u003ca href=\"examples/demo.py\"\u003e\u003ccode\u003epython examples/demo.py\u003c/code\u003e\u003c/a\u003e.\u003c/sub\u003e\n\u003c/p\u003e\n\nApple exposes the Neural Engine only through CoreML, for inference only.\nCoreML decides whether your model lands on the engine or quietly falls back to the\nCPU or GPU, and it gives you no way to train there. ANEForge skips it: it compiles\na tensor graph into one ANE program and dispatches that program through the same\nprivate `aned` stack CoreML, MPSGraph, and Espresso use internally. From there:\n\n- **Training runs on the engine.** The forward pass, the backward pass, and the Adam update all compile to ANE programs.\n- A CNN trains from scratch on CIFAR-10 to 71%, on a chip Apple ships for inference only.\n- **Hardware layers CoreML can't reach.** `af.sdpa` drives the engine's fused-attention layer directly, the one Apple's compiler decomposes and never emits; 18 other native layers (`argmax`, `topk`, `sort`, geometry) come the same way.\n- **The engine, never a fallback.** A pretrained ResNet-18 runs end-to-end in 0.33 ms, matching the reference to cosine 1.0000, at a fraction of the GPU's energy (table below).\n- **Cross-compilation for chips you don't own.** Lower and gate a graph for any of 28 ANE targets (M1-M5) from one machine, and estimate its latency without running it.\n\n```python\nimport aneforge as af\n\nx   = af.input((1, 3, 32, 32))             # a lazy graph input\ny   = af.conv(x, W, pad=1).relu().mean((2, 3))\nnet = af.compile(y, compress=\"int8\")       # graph -\u003e one fused ANE program\nout = net(image)                           # callable; runs on ANE silicon\n\n# ...or load a pretrained model\nenc = af.load(\".../all-MiniLM-L6-v2\")      # MiniLM sentence encoder\nvec = enc(tokens)                          # on-device, cosine 1.0000 vs reference\n```\n\nA graph is built from 58 fused operators and 19 native bridge operators, lowered into a single program and reused across calls, with a near-70 us dispatch floor.\n\n\u003e **Status:** research project on Apple Silicon / macOS, verified on M5 Pro and M1\n\u003e Max. Relies on private framework symbols that may change without notice. Not\n\u003e affiliated with Apple.\n\n## Install\n\nApple Silicon Mac, macOS 14+, Xcode command-line tools, Python 3.10+.\n\n```sh\npip install aneforge\n```\n\nThe `e5rt` dispatch shim links Apple frameworks, so it compiles from source on your Mac the first time you dispatch to the ANE (or ahead of time with `python -m aneforge.build`).\nOptional extras: `pip install \"aneforge[models]\"` for the pretrained loaders (torch / torchvision / transformers).\n\nFor the examples, tests, and benchmarks, work from a checkout:\n\n```sh\ngit clone https://github.com/sbryngelson/ANEForge.git\ncd ANEForge\npip install -e \".[dev]\"\nPYTHONPATH=. python3 tests/op_smoketest.py    # compile + run each op on the ANE\n```\n\nThen browse [`examples/`](examples/), starting with\n[`examples/quickstart.py`](examples/quickstart.py).\n\n## How it compares\n\n|                       | On the ANE        | No CoreML | Trains on it |\n| --------------------- | :---------------: | :-------: | :----------: |\n| CoreML / coremltools  | scheduler chooses | --        | no           |\n| MLX, PyTorch (MPS)    | no (GPU)          | yes       | on the GPU   |\n| **ANEForge**          | **yes (direct)**  | **yes**   | **yes**      |\n\nCoreML is the only public door to the engine, and it only ever decides whether to\nuse it. ANEForge compiles to the engine directly, from an ordinary user process,\nwith no entitlement and without disabling system integrity protection.\n\n## Measured\n\nSingle input, fp16, on an M5 Pro. The GPU baseline is PyTorch on Metal (MPS) at\nfp16; energy is whole-package, read with `powermetrics`.\n\n| Pretrained model | ANE     | GPU (fp16) | ANE energy | GPU energy |\n| ---------------- | ------: | ---------: | ---------: | ---------: |\n| ResNet-18        | 0.33 ms | 2.03 ms    | 2.2 mJ     | 35 mJ      |\n| MiniLM encoder   | 0.53 ms | 1.92 ms    | 2.4 mJ     | 21 mJ      |\n| ViT-B/16         | 18.3 ms | 15.9 ms    | 75 mJ      | 612 mJ     |\n\nThe engine is faster on the convolutional and encoder workloads and 8-16x more energy-efficient on all three, even on ViT-B/16, where the GPU edges it in latency.\nReproduce with [`bench/device_compare_wattcomplete.py`](bench/device_compare_wattcomplete.py) and [`bench/real_models_fp16.py`](bench/real_models_fp16.py); the full per-workload device map (16 classes, measured on M1 / M2 / M5) is in [`bench/results/`](bench/results/).\n\n## A fluid simulation on the Neural Engine\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/assets/fluid_vorticity.png\" width=\"400\"\n       alt=\"A passive dye shaped as the word ANEForge stirred into glowing filaments by a fluid simulation on the Apple Neural Engine\"\u003e\n\u003c/p\u003e\n\nA passive dye is painted as the word ANEForge, and a 2-D incompressible Navier-Stokes flow (pseudo-spectral) stirs it into thin glowing filaments.\nEvery Fourier transform in the 2,200-step loop runs on the ANE, and the whole simulation costs about 9 J at the measured 1.48 W rail.\nReproduce with [`python examples/fluid_vorticity.py`](examples/fluid_vorticity.py).\n\n## What it does\n\n- **Graph -\u003e compile -\u003e run.** 58 fused operators (conv/pool, `matmul`/`bmm`/`einsum`, activations, reductions, norms, softmax, attention, shape/geometry) into one program with int8/int4/fp16 weights, plus a bridge route for 19 native ops the public toolchain never emits.\n- **Streaming weight compression.** int8, int4-LUT, or sparse weights streamed from the engine's dequant path (~4x smaller for int4), accuracy-gated.\n- **On-device uint8 image input,** dequantized in-graph, so raw camera or video bytes feed the model directly.\n- **Resident state.** KV-cache and optimizer state kept on the engine across steps via buffer aliasing (`share_buffer`).\n- **Accuracy-preserving optimizer.** `af.tune` measures equivalent lowerings on the engine and returns the lossless pick.\n- **Linear algebra and spectral methods.** `aneforge.linalg` and `aneforge.fft` as static-dataflow graphs.\n\n## What runs\n\nPretrained models, each fused into one ANE program:\n\n| Model              | Task                       | Fidelity vs reference   |\n| ------------------ | -------------------------- | ----------------------- |\n| ResNet-18          | ImageNet classification    | cosine 1.0000           |\n| ViT-B/16           | vision transformer encoder | cosine 1.0000           |\n| all-MiniLM-L6-v2   | sentence embedding         | cosine 1.0000           |\n| ESPCN              | super-resolution           | runs end to end         |\n| Stable Diffusion 1.5 | U-Net + VAE (per component) | U-Net 1.5%, VAE 4.4% rel. |\n\nTrained from scratch on the engine: an MLP, a CNN (CIFAR-10 to 71%), a transformer block, a LLaMA-style block, and a character language model.\nOperator coverage is tracked op by op across M1 to M5 in the [op catalog](docs/op-catalog.md), the exhaustive native-MIL-op x device table; [capabilities](docs/capabilities.md) has the dtype matrix and the known limits.\n\n## Verify\n\nThe correctness corpus compiles and runs every op and kernel on the ANE, and serves as a reproducibility test:\n\n```sh\nPYTHONPATH=. python3 tests/run_corpus.py\nPYTHONPATH=. python3 -m pytest tests/ -q\n```\n\n## Documentation\n\nThe manual is hosted at [aneforge.readthedocs.io](https://aneforge.readthedocs.io).\nThe API is documented in the module docstrings and demonstrated in [`examples/`](examples/).\n\n## Contributing\n\n[`CONTRIBUTING.md`](CONTRIBUTING.md) has the bug-report checklist (include your chip and macOS version), the development setup, and where to start.\nReport security issues privately per the [`SECURITY.md`](SECURITY.md) guidelines.\n\n## License\n\n[MIT](LICENSE). The Apple Neural Engine is proprietary hardware, and the framework symbols this project calls are private, undocumented, and may change at any time.\nNothing here is endorsed by, or constitutes an API contract from, Apple.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsbryngelson%2Faneforge","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsbryngelson%2Faneforge","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsbryngelson%2Faneforge/lists"}