{"id":49005684,"url":"https://github.com/hankthebldr/local-ai-platform","last_synced_at":"2026-05-31T23:00:53.352Z","repository":{"id":331149809,"uuid":"1111997840","full_name":"hankthebldr/local-ai-platform","owner":"hankthebldr","description":"Self-hosted LLM infrastructure with OpenAI-compatible API. CPU-optimized. Source-available.","archived":false,"fork":false,"pushed_at":"2026-05-27T16:50:09.000Z","size":31511,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2026-05-27T17:13:32.657Z","etag":null,"topics":["ai-inference","cpu-inference","enclave","llm","local-ai","macos","ollama","openai-compatible","privacy","self-hosted"],"latest_commit_sha":null,"homepage":"https://hankthebldr.github.io/local-ai-platform/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hankthebldr.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-08T02:11:36.000Z","updated_at":"2026-05-27T16:40:01.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/hankthebldr/local-ai-platform","commit_stats":null,"previous_names":["hankthebldr/local-ai-platform"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/hankthebldr/local-ai-platform","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hankthebldr%2Flocal-ai-platform","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hankthebldr%2Flocal-ai-platform/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hankthebldr%2Flocal-ai-platform/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hankthebldr%2Flocal-ai-platform/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hankthebldr","download_url":"https://codeload.github.com/hankthebldr/local-ai-platform/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hankthebldr%2Flocal-ai-platform/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33752286,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-05-31T02:00:06.040Z","response_time":95,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-inference","cpu-inference","enclave","llm","local-ai","macos","ollama","openai-compatible","privacy","self-hosted"],"created_at":"2026-04-18T20:07:16.578Z","updated_at":"2026-05-31T23:00:53.346Z","avatar_url":"https://github.com/hankthebldr.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/logo/enclave-mark.svg\" width=\"80\" alt=\"Enclave\"\u003e\n\u003c/p\u003e\n\n\u003ch1 align=\"center\"\u003eEnclave\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003e\n  Self-hosted LLM infrastructure with OpenAI-compatible API. CPU-optimized. Source-available.\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/hankthebldr/local-ai-platform/releases/latest\"\u003e\u003cimg src=\"https://img.shields.io/github/v/release/hankthebldr/local-ai-platform?label=release\u0026labelColor=1a1a2e\u0026color=00E87B\u0026style=flat\" alt=\"Latest release\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/hankthebldr/local-ai-platform/releases?q=prerelease%3Atrue\u0026expanded=true\"\u003e\u003cimg src=\"https://img.shields.io/github/v/release/hankthebldr/local-ai-platform?include_prereleases\u0026label=nightly\u0026labelColor=1a1a2e\u0026color=FA582D\u0026style=flat\" alt=\"Nightly\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/hankthebldr/local-ai-platform/actions/workflows/ci.yml\"\u003e\u003cimg src=\"https://img.shields.io/github/actions/workflow/status/hankthebldr/local-ai-platform/ci.yml?branch=master\u0026labelColor=1a1a2e\u0026color=00C0E8\u0026style=flat\u0026label=ci\" alt=\"CI\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://hub.docker.com/r/hankthebldrr/local-ai-platfrom\"\u003e\u003cimg src=\"https://img.shields.io/docker/pulls/hankthebldrr/local-ai-platfrom?labelColor=1a1a2e\u0026color=00C0E8\u0026style=flat\u0026label=docker%20pulls\" alt=\"Docker pulls\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/hankthebldr/local-ai-platform/pkgs/container/enclave\"\u003e\u003cimg src=\"https://img.shields.io/badge/ghcr.io-enclave-1a1a2e?style=flat\u0026labelColor=1a1a2e\u0026color=00C0E8\" alt=\"GHCR\"\u003e\u003c/a\u003e\n  \u003cimg src=\"https://img.shields.io/badge/macOS%2012%2B-supported-1a1a2e?style=flat\u0026labelColor=1a1a2e\u0026color=00C0E8\" alt=\"macOS\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/Linux-supported-1a1a2e?style=flat\u0026labelColor=1a1a2e\u0026color=00C0E8\" alt=\"Linux\"\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://hankthebldr.github.io/local-ai-platform/\"\u003e\u003cstrong\u003eProduct page\u003c/strong\u003e\u003c/a\u003e ·\n  \u003ca href=\"https://github.com/hankthebldr/local-ai-platform/wiki\"\u003e\u003cstrong\u003eWiki\u003c/strong\u003e\u003c/a\u003e ·\n  \u003ca href=\"https://github.com/hankthebldr/local-ai-platform/releases/latest\"\u003e\u003cstrong\u003eLatest release\u003c/strong\u003e\u003c/a\u003e ·\n  \u003ca href=\"CHANGELOG.md\"\u003e\u003cstrong\u003eChangelog\u003c/strong\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n---\n\nEnclave runs LLMs on your hardware. OpenAI-compatible API, Ollama backend, zero cloud dependencies.\n\n\u003e **What's new** — Architecture-aware orchestration (Phases 1–6): per-host detection of memory + deployment topology, four-tier `keep_alive` resolver with arch-detected defaults, scheduler facade with feasibility validation, and tick-based **parallel DAG dispatch** that uses the arch to decide what to run concurrently. Plus: installable Python wheel + sdist, mirrored Docker image on GHCR, Linux source tarball with SHA256/SHA512, n8n release-update workflow, and a curated Wiki seed. See the [CHANGELOG](CHANGELOG.md) for the full PR-by-PR detail.\n\n## What it does\n\n- **OpenAI-compatible API** — drop-in replacement. Point your existing code at `localhost:8000`\n- **CPU-optimized inference** — GGUF quantized models via Ollama. 7B at 40-50 tok/s, 13B at 25-30 tok/s\n- **Model management** — download, configure, and switch between 18+ models from the registry\n- **Multi-agent workflows** — YAML-defined step pipelines with role-based model selection\n- **Web dashboard** — monitor models, system health, and API status\n- **macOS app** — native desktop wrapper with setup wizard\n- **No telemetry by default** — no data leaves your machine unless you opt in; optional, operator-owned error reporting (your own sink, redaction mandatory — see [docs/deployment/error-reporting.md](docs/deployment/error-reporting.md)). No internet required for inference\n\n## Quick start\n\nThree paths — pick one:\n\n### macOS app (DMG) — for end users\n\n1. Download **Enclave.dmg** from the [latest release](https://github.com/hankthebldr/local-ai-platform/releases/latest)\n   *(Or grab the rolling [nightly build](https://github.com/hankthebldr/local-ai-platform/releases/tag/nightly) for the freshest master.)*\n2. Open the DMG and drag **Enclave.app** to `/Applications`.\n3. First launch: macOS Gatekeeper will warn — the app is currently **not signed/notarized**. Bypass once with:\n   ```bash\n   xattr -dr com.apple.quarantine /Applications/Enclave.app\n   ```\n   Then double-click **Enclave** in Launchpad.\n4. The native window opens the **first-run setup wizard** (`/setup`) which installs Ollama if needed and pulls a starter model. After that you land on the dashboard.\n\n\u003e **Requirements:** macOS 12.0 (Monterey) or later. ~6 GB free disk for the bundled runtime + a small starter model. Ollama is installed automatically by the wizard if missing.\n\n### Docker — any platform with Docker Desktop\n\nFor non-developers on Linux / Windows, or anyone who wants Enclave fully isolated in containers. No Python, no virtualenv, no manual Ollama install.\n\n1. Install [Docker Desktop](https://www.docker.com/products/docker-desktop/) (or Docker Engine on Linux) and make sure the whale icon is running.\n2. Clone or download this repo, open a terminal in the project folder, and run:\n   ```bash\n   ./run.sh\n   ```\n3. The script verifies Docker, brings up the stack (`ollama` + `api`), pulls a small starter model on first run (`llama3.2:3b`, ~2 GB), and opens the dashboard in your browser.\n\n| | URL |\n|---|---|\n| **Enclave SPA** (the application) | `http://localhost:8000` |\n| API docs                          | `http://localhost:8000/docs` |\n| Open WebUI (opt-in)               | `http://localhost:8081` — `docker compose -f docker-compose.yml -f docker-compose.webui.yml up -d` |\n\nTo stop: `./stop.sh` (data preserved) — or `./stop.sh --reset` to wipe models and chat history.\n\n\u003e **Requirements:** ~4 GB free RAM and ~3 GB free disk for the starter model. Pick a different starter with `ENCLAVE_DEFAULT_MODEL=qwen2.5:3b ./run.sh`.\n\nPrefer to pull the published image directly? (Substitute `\u003cversion\u003e` with the latest tag.)\n\n```bash\n# Docker Hub — canonical\ndocker pull hankthebldrr/local-ai-platfrom:\u003cversion\u003e\n\n# GHCR mirror — same digest, no Hub account required\ndocker pull ghcr.io/hankthebldr/enclave:\u003cversion\u003e\n```\n\n### pip install — embed in an existing Python app\n\nFor developers who want to use the Enclave engine inside another Python service. Bundles the FastAPI app, workflow engine, RAG pipeline, and CLI dispatcher.\n\n```bash\n# From a GitHub Release asset (no PyPI required)\npip install https://github.com/hankthebldr/local-ai-platform/releases/download/v\u003cversion\u003e/enclave-\u003cversion\u003e-py3-none-any.whl\n\n# Then run the API server with the same uvicorn settings the DMG uses:\nenclave-api                 # starts FastAPI on 127.0.0.1:8000\nenclave --help              # CLI dispatcher (chat, workflow, query, api)\n```\n\nYou still need an Ollama runtime reachable at `OLLAMA_URL` (defaults to `http://localhost:11434`). The Python package does **not** install Ollama for you — see the [Wiki › Deployment](https://github.com/hankthebldr/local-ai-platform/wiki/Deployment) page for production setups.\n\n### From source — for developers\n\n```bash\n# Install (creates ./venv, installs core+dev deps, sets up systemd unit on Linux)\n./setup/install.sh\n\n# Boot Ollama + API + auto-open the dashboard in your browser\n./scripts/start.sh\n\n# Or, on macOS, exercise the same native pywebview window the DMG ships\n./scripts/start_desktop.sh\n\n# Verify everything boots and every UX route renders\n./scripts/verify_local.sh\n```\n\nAPI at `http://localhost:8000` · Dashboard at `http://localhost:8000/` · Docs at `http://localhost:8000/docs` · First-run wizard at `http://localhost:8000/setup`.\n\n## Models\n\n```bash\n# List available models\npython models/download.py --list\n\n# Download a model\npython models/download.py dolphin-mixtral\n\n# List installed\nollama list\n```\n\nDefault quantization: Q4_K_M (best quality/speed balance). See [MODELS.md](MODELS.md) for the full registry.\n\n## API usage\n\n```bash\ncurl http://localhost:8000/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"model\": \"mistral\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"Hello\"}]\n  }'\n```\n\nCompatible with any OpenAI SDK client.\n\n## Code-level artifacts\n\nWhat ships in this repo, and where to find it:\n\n| Surface | Path | Notes |\n|---|---|---|\n| FastAPI server (OpenAI-compatible) | [api/main.py](api/main.py) | 16 routers under `api/routers/`, services under `api/services/` |\n| Web dashboard + setup wizard | [api/static/](api/static/) | Served at `/` and `/setup` by the FastAPI app |\n| CLI chat / query / workflow | [cli/](cli/) | Rich-formatted; `python -m cli.chat`, `cli/workflow.py` |\n| Multi-agent workflow engine | [api/services/workflow_engine.py](api/services/workflow_engine.py) | YAML pipelines under [workflows/](workflows/) |\n| Custom agents (Gems) | [agents/](agents/) + [api/routers/agents.py](api/routers/agents.py) | YAML-defined personas with pinned context |\n| Model registry | [models/download.py](models/download.py) | 18+ models — see [MODELS.md](MODELS.md) |\n| macOS desktop wrapper | [desktop/app.py](desktop/app.py) | pywebview window around the FastAPI server |\n| DMG builder | [scripts/build_mac.sh](scripts/build_mac.sh) | Bundles a self-contained `.app` + dmg |\n| Local dev scripts | [scripts/](scripts/) | `start.sh`, `start_desktop.sh`, `verify_local.sh`, `status.sh`, `test.sh` |\n\n### Build the DMG yourself\n\nThe same script CI uses on tag pushes:\n\n```bash\nbrew install librsvg create-dmg     # one-time\n./scripts/generate-icons.sh         # regenerate icns from SVG\n./scripts/build_mac.sh              # produces dist/Enclave.app + dist/Enclave.dmg\nopen dist/Enclave.app               # smoke-test the bundle\n```\n\nThe build script reads `ENCLAVE_VERSION` (or falls back to `git describe`) and stamps it into `Info.plist`. Override for a one-off custom build:\n\n```bash\nENCLAVE_VERSION=v1.2.3-local ./scripts/build_mac.sh\n```\n\n### Release pipeline\n\n| Trigger | Workflow | Artifact |\n|---|---|---|\n| Tag push `v*.*.*` | [release.yml](.github/workflows/release.yml) | Stable GitHub Release with signed DMG |\n| Push to `master` | [release.yml](.github/workflows/release.yml) | Rolling `nightly` pre-release (replaced each merge) |\n| PR / push to `master` | [ci.yml](.github/workflows/ci.yml) | pytest + lint + macOS `.app` smoke build (boots and probes UX routes) |\n| Tag push or release publish | [pages.yml](.github/workflows/pages.yml) | Updates [hankthebldr.github.io/local-ai-platform](https://hankthebldr.github.io/local-ai-platform/) with the latest release version |\n\nEvery master merge re-publishes a freshly smoke-tested DMG to the [`nightly`](https://github.com/hankthebldr/local-ai-platform/releases/tag/nightly) release. Stable releases are cut by pushing a `vX.Y.Z` tag.\n\n## Hardware targets\n\n| Machine | RAM | Role | Throughput |\n|---------|-----|------|------------|\n| Mac M4 Pro | 48GB | Development | 7B @ 50 tok/s |\n| MS-01 (Ryzen 9 7945HX) | 64GB | API serving | 34B @ 12 tok/s |\n| BD790i (Ryzen 9 7945HX) | 96GB | Research / 70B-class workflows | 70B @ 5 tok/s |\n\nThe BD790i is the only host in the fleet that can exercise the full\n1.3.0 MCP \u0026 Skills co-scheduler against 70B-class models + multi-GB\nMCP RSS simultaneously. Bring-up + benchmark recipes:\n[docs/deployment/bd790i-testing.md](docs/deployment/bd790i-testing.md).\n\n## Documentation\n\nThe canonical operator-facing docs live on the [GitHub Wiki](https://github.com/hankthebldr/local-ai-platform/wiki) (sourced from [docs/wiki/](docs/wiki/) on every tag). Highlights:\n\n- [Quickstart](https://github.com/hankthebldr/local-ai-platform/wiki/Quickstart) — first 60 seconds\n- [Architecture](https://github.com/hankthebldr/local-ai-platform/wiki/Architecture) — request flow, services, workflow engine, arch-aware dispatch\n- [Workflows](https://github.com/hankthebldr/local-ai-platform/wiki/Workflows) — authoring YAML pipelines + composite step kinds\n- [Agents](https://github.com/hankthebldr/local-ai-platform/wiki/Agents) — Gems-style YAML personas\n- [Models](https://github.com/hankthebldr/local-ai-platform/wiki/Models) — registry, quantization, throughput\n- [Deployment](https://github.com/hankthebldr/local-ai-platform/wiki/Deployment) — DMG · Docker · pip · source · systemd\n- [Configuration](https://github.com/hankthebldr/local-ai-platform/wiki/Configuration) — env vars, auth, CORS, perf knobs\n- [Troubleshooting](https://github.com/hankthebldr/local-ai-platform/wiki/Troubleshooting) — common failure modes\n- [Release notes](https://github.com/hankthebldr/local-ai-platform/wiki/Release-Notes)\n\nSource-of-truth references inside the repo:\n\n- [MODELS.md](MODELS.md) — model registry and selection\n- [CLAUDE.md](CLAUDE.md) — developer guide\n- [CHANGELOG.md](CHANGELOG.md) — every release, every PR\n- [docs/](docs/) — design docs, plans, deployment guides\n- Product page: [hankthebldr.github.io/local-ai-platform](https://hankthebldr.github.io/local-ai-platform/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhankthebldr%2Flocal-ai-platform","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhankthebldr%2Flocal-ai-platform","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhankthebldr%2Flocal-ai-platform/lists"}