{"id":48956673,"url":"https://github.com/stampby/halo-ai-core","last_synced_at":"2026-04-18T00:09:25.627Z","repository":{"id":349905511,"uuid":"1204444812","full_name":"stampby/halo-ai-core","owner":"stampby","description":"Bare-metal AI platform for AMD Strix Halo. One script. Everything works. Lego blocks — snap in what you need.","archived":false,"fork":false,"pushed_at":"2026-04-15T23:13:13.000Z","size":929,"stargazers_count":34,"open_issues_count":0,"forks_count":4,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-16T00:19:21.200Z","etag":null,"topics":["agent-framework","ai","amd","arch-linux","bare-metal","caddy","gaia","gpu","inference","lemonade","llama-cpp","local-ai","privacy","rocm","ryzen-ai","self-hosted","strix-halo","systemd"],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stampby.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"docs/SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-08T02:37:35.000Z","updated_at":"2026-04-15T23:58:13.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/stampby/halo-ai-core","commit_stats":null,"previous_names":["stampby/halo-ai-core"],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/stampby/halo-ai-core","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stampby%2Fhalo-ai-core","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stampby%2Fhalo-ai-core/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stampby%2Fhalo-ai-core/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stampby%2Fhalo-ai-core/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stampby","download_url":"https://codeload.github.com/stampby/halo-ai-core/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stampby%2Fhalo-ai-core/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31950895,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-17T17:29:20.459Z","status":"ssl_error","status_checked_at":"2026-04-17T17:28:47.801Z","response_time":62,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agent-framework","ai","amd","arch-linux","bare-metal","caddy","gaia","gpu","inference","lemonade","llama-cpp","local-ai","privacy","rocm","ryzen-ai","self-hosted","strix-halo","systemd"],"created_at":"2026-04-18T00:09:25.098Z","updated_at":"2026-04-18T00:09:25.612Z","avatar_url":"https://github.com/stampby.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n🌐 **English** | [Français](README.fr.md) | [Español](README.es.md) | [Deutsch](README.de.md) | [Português](README.pt.md) | [日本語](README.ja.md) | [中文](README.zh.md) | [한국어](README.ko.md) | [Русский](README.ru.md) | [हिन्दी](README.hi.md) | [العربية](README.ar.md)\n\n# halo-ai core\n\n### the 1-bit monster — local ai inference, bare metal, no python at runtime\n\n**rocm c++ · ternary weights (.h1b) · fused HIP kernels · wave32 wmma · 17 c++ specialists · zero telemetry · zero cloud**\n\n*stamped by the architect*\n\n[![License](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)\n[![ROCm](https://img.shields.io/badge/ROCm_7.13-ED1C24?style=flat\u0026logo=amd\u0026logoColor=white)](https://github.com/ROCm/TheRock)\n[![rocm-cpp](https://img.shields.io/badge/rocm--cpp-inference_engine-00d4ff?style=flat)](https://github.com/stampby/rocm-cpp)\n[![agent-cpp](https://img.shields.io/badge/agent--cpp-17_specialists-00d4ff?style=flat)](https://github.com/stampby/agent-cpp)\n[![halo-1bit](https://img.shields.io/badge/halo--1bit-.h1b_format-00d4ff?style=flat)](https://github.com/stampby/halo-1bit)\n[![Discord](https://img.shields.io/badge/Discord-halo--ai-5865F2?style=flat\u0026logo=discord\u0026logoColor=white)](https://discord.gg/dSyV646eBs)\n[![Reddit](https://img.shields.io/badge/Reddit-r/MidlifeCrisisAI-FF4500?style=flat\u0026logo=reddit\u0026logoColor=white)](https://www.reddit.com/r/MidlifeCrisisAI/)\n[![Self Hosted](https://img.shields.io/badge/Self_Hosted-100%25_Local-purple?style=flat)](https://github.com/stampby/halo-ai-core)\n\n\u003c/div\u003e\n\n---\n\n## what is this\n\nhalo-ai core is the **install script for the 1-bit monster** — a full local AI stack that runs entirely in C++ on AMD Strix Halo hardware. no python at runtime. no cloud. no telemetry. no subscriptions.\n\none script, three engineering repos:\n\n| repo | what it is |\n|------|-----------|\n| [**rocm-cpp**](https://github.com/stampby/rocm-cpp) | the inference engine. pure HIP, fused ternary kernels, OpenAI-compatible server with SSE streaming. |\n| [**agent-cpp**](https://github.com/stampby/agent-cpp) | the agent framework. 17 single-purpose specialists on a message bus, hash-chained audit log, consent-verification gate. |\n| [**halo-1bit**](https://github.com/stampby/halo-1bit) | the model format (.h1b) + training pipeline. absmean ternary, QAT with straight-through estimator, distillation from bf16 teachers. |\n\nhalo-ai core clones them, builds them from source, wires them into systemd, and points a caddy reverse proxy at the result. one command, you get a running LLM, a voice loop, a discord bot, a CI runner, and an audit trail. everything local.\n\n*\"I know kung fu.\"*\n\n## install\n\ntwo paths. the script auto-detects your GPU and picks the right one.\n\n```bash\ngit clone https://github.com/stampby/halo-ai-core.git\ncd halo-ai-core\n./install.sh                  # auto-dispatch: strixhalo → fast; else → source\n```\n\n| path | who it's for | time | what it does |\n|------|--------|------|------|\n| [`./install-strixhalo.sh`](install-strixhalo.sh) | **gfx1151** (Strix Halo) | ~5 min | downloads pre-built binaries from GH Releases, verifies SHA256 + GPG, wires systemd |\n| [`./install-source.sh`](install-source.sh) | any other AMD GPU | ~4 hrs | builds TheRock + rocm-cpp + agent-cpp + halo-1bit from source for your arch |\n\nwhy two scripts: every Strix Halo is the same silicon (gfx1151, wave32, 128 GB unified). one build produces a binary that runs bit-identically on every such box — no reason to rebuild from source every time. for anything else (gfx1030, gfx1100, gfx1201, CDNA), the wave32 WMMA kernels don't port 1:1, so source build with arch-specific codegen is the safe option.\n\n**running something other than a strix halo and want the kernels built for your GPU?** see [`release/KERNELS.md`](release/KERNELS.md) for arch coverage, how to build your own, and how to share community builds back.\n\n[![Install Demo](https://img.shields.io/badge/asciinema-watch_install_demo-d40000?style=flat\u0026logo=asciinema\u0026logoColor=white)](docs/install-rocmpp.cast)\n\n## the stack\n\n```\n┌─────────────────────────────────────────────────────────┐\n│            agent-cpp — 17 C++ specialists                │\n│   muse · planner · forge · warden (CVG) · scribe         │\n│   sommelier · herald · sentinel · carpenter · anvil      │\n│   quartermaster · magistrate · librarian · cartograph    │\n│   echo_ear · echo_mouth · stdout_sink                    │\n├─────────────────────────────────────────────────────────┤\n│  rocm-cpp server (:8080) — OpenAI-compat, SSE streaming  │\n├─────────────────────────────────────────────────────────┤\n│   librocm_cpp — HIP kernels · WMMA wave32 · KV cache    │\n├─────────────────────────────────────────────────────────┤\n│  ternary model (.h1b v2)  ·  halo-1bit tokenizer (.htok) │\n├─────────────────────────────────────────────────────────┤\n│          whisper-server (STT) · kokoro (TTS)             │\n├─────────────────────────────────────────────────────────┤\n│              ROCm 7.13.0  ·  gfx1151 wave32              │\n├─────────────────────────────────────────────────────────┤\n│              Arch Linux · systemd · btrfs                │\n└─────────────────────────────────────────────────────────┘\n```\n\n\u003e *every layer is someone else's lego block if they want it. take the whole monster or take one piece.*\n\n## numbers that matter\n\n| metric | value | note |\n|---|---|---|\n| **decode speed** | 85 tok/s | BitNet-b1.58-2B, greedy, Strix Halo |\n| **model size** | 1.1 GiB | TQ1_0 format, 4× smaller than F16 |\n| **KLD vs F16** | 0.0023 | mean bits/token — indistinguishable in practice |\n| **top-1 agreement** | 96.3% | vs F16 reference, same argmax token |\n| **agent binary** | 1.3 MB | agent_cpp, statically linked |\n| **cold start** | \u003c 2s | bitnet_decode --server |\n| **runtime deps** | 0 python | libc, pthreads, httplib, nlohmann-json, OpenSSL |\n\ndetails and methodology: [docs/benchmark-comparison.md](docs/benchmark-comparison.md) · [docs/replicate.md](docs/replicate.md)\n\n## what you get\n\n### the engine\n- **bitnet_decode** — OpenAI-compatible HTTP server on :8080 with SSE streaming. chat completions, models list, bearer auth optional.\n- **`.h1b` loader** — ternary weights, magic `H1B`, 9 int32 config + 2 float32 params. memory-mapped, zero-copy.\n- **HIP kernels** — fused ternary MatMul, RMSNorm, SiLU, RoPE. wave32 WMMA. no CK, no hipBLAS at runtime.\n\n### the agents\n- **17 specialists** — each one job, each one thread. message bus with tamper-evident journal.\n- **consent-verification gate** — warden enforces policy/intent/consent/bounds. structural, not advisory.\n- **hash-chained audit log** — every inbound and outbound message SHA-256 chained, genesis-seeded per session.\n- **optional plugins** — Discord read (sentinel) + write (herald), GitHub triage/PR-review/docs (quartermaster/magistrate/librarian), CI runner (anvil), install-help (carpenter).\n\n### the model + training\n- **halo-1bit** — absmean quantization, QAT with STE, distillation from Qwen3-32B bf16 teacher.\n- **.h1b v2 format** — production artifacts shipped with each release.\n\n## lego blocks\n\npick what you want. drop the rest.\n\n| block | what it does | status |\n|-------|------|--------|\n| **bitnet_decode** | inference server | required |\n| **agent_cpp** | agent framework | required |\n| **agent_cpp → sentinel+herald** | Discord bot | optional (set DISCORD_TOKEN) |\n| **agent_cpp → echo_ear+echo_mouth** | voice loop | optional (whisper + kokoro services) |\n| **agent_cpp → quartermaster/magistrate/librarian** | GitHub automation | optional (set GH_TOKEN) |\n| **agent_cpp → anvil** | CI runner | optional |\n| **caddy** | reverse proxy + bearer auth | optional |\n| **man-cave TUI** | FTXUI dashboard over SSH | optional (v2) |\n| **orchestrator** | systemd unit wiring | included |\n\n## philosophy\n\n\u003e every piece snaps in and snaps out. no hard dependencies. no vendor lock-in. no cloud tethers.\n\npython shipped the LLM era. C++ owns the next one. python at training time is fine; python at runtime is a liability on hardware you own. **halo-ai core has zero python at runtime.**\n\nthe AI industry wants you renting someone else's computer. we think you should own the whole stack — the hardware, the models, the weights, the pipeline. when you control your own software, you control your own destiny.\n\n*\"they get the kingdom. they forge their own keys.\"*\n\n## privacy\n\n**zero telemetry. zero tracking. zero data collection.** nothing phones home. your data stays on your machine.\n\npaid API providers (OpenAI, Anthropic, Groq, DeepSeek, xAI, OpenRouter) are supported through sommelier with your own keys — but that's your choice, not our default. local-first means local-first.\n\n*\"there is no cloud. there is only zuul.\"*\n\n## docs\n\n| doc | what it covers |\n|-----|---|\n| [docs/INTEGRATIONS.md](docs/INTEGRATIONS.md) | **point your apps at the stack — openai sdk, curl, python, node, c++, webui, mobile** |\n| [docs/benchmark-comparison.md](docs/benchmark-comparison.md) | reproducible numbers vs llama.cpp / vLLM / MLX |\n| [docs/replicate.md](docs/replicate.md) | step-by-step: build the monster on your box |\n| [docs/mlx-setup-guide.md](docs/mlx-setup-guide.md) | the MLX path (comparison / optional) |\n| [orchestrator/README.md](orchestrator/README.md) | systemd unit wiring |\n| [prototypes/](prototypes/) | next-rung experiments (ternary dequant, etc.) |\n| [docs/archive/](docs/archive/) | legacy wiki + pre-monster docs |\n\n## options\n\n```\n./install.sh --dry-run         preview without installing\n./install.sh --yes-all         install everything\n./install.sh --status          check what's running\n./install.sh --skip-\u003cblock\u003e    skip any optional block\n./install.sh --help            all options\n```\n\n## requirements\n\n- Arch Linux (bare metal preferred; podman works for headless)\n- AMD Ryzen AI hardware — Strix Halo (gfx1151) or Strix Point (gfx1150)\n- passwordless sudo\n- ~20 GiB free disk (build artifacts, kernels, models)\n\n## credits\n\nthis project stands on the shoulders of the people who ship open source.\n\nbuilt on [llama.cpp](https://github.com/ggml-org/llama.cpp) (for eval tooling), [TheRock](https://github.com/ROCm/TheRock) (ROCm distribution), [httplib](https://github.com/yhirose/cpp-httplib), [nlohmann/json](https://github.com/nlohmann/json), [usearch](https://github.com/unum-cloud/usearch), [FTXUI](https://github.com/ArthurSonzogni/FTXUI), [whisper.cpp](https://github.com/ggerganov/whisper.cpp), [Kokoro TTS](https://github.com/remsky/Kokoro-FastAPI), [microsoft/bitnet-b1.58-2B-4T](https://huggingface.co/microsoft/bitnet-b1.58-2B-4T) (reference model).\n\nspecial thanks to the `r/MidlifeCrisisAI` community for hard questions, especially the ones about PPL and KLD.\n\n---\n\n\u003cdiv align=\"center\"\u003e\n\n*\"the 1-bit monster is already here. it just had to learn to count.\"* — **stamped by the architect**\n\nMIT\n\n\u003c/div\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstampby%2Fhalo-ai-core","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstampby%2Fhalo-ai-core","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstampby%2Fhalo-ai-core/lists"}