{"id":31556988,"url":"https://github.com/BingoWon/orchardgrid-apple","last_synced_at":"2026-04-15T04:02:12.282Z","repository":{"id":317573077,"uuid":"1067991984","full_name":"BingoWon/orchardgrid-app","owner":"BingoWon","description":"OpenAI-Compatible API Server for Apple Intelligence - Structured Output, Streaming, Multi-turn Conversations","archived":false,"fork":false,"pushed_at":"2025-10-02T23:38:31.000Z","size":45,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-03T01:24:48.755Z","etag":null,"topics":["api-server","apple-intelligence","foundation-models","json-schema","macos","openai-api","structured-output","swift","swiftui"],"latest_commit_sha":null,"homepage":null,"language":"Swift","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BingoWon.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-01T17:31:50.000Z","updated_at":"2025-10-02T23:38:34.000Z","dependencies_parsed_at":"2025-10-03T01:24:59.458Z","dependency_job_id":"acb3ad85-65d9-4d28-9633-ba7231113d7e","html_url":"https://github.com/BingoWon/orchardgrid-app","commit_stats":null,"previous_names":["bingowon/giant_big","bingowon/orchardgrid-app"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/BingoWon/orchardgrid-app","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BingoWon%2Forchardgrid-app","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BingoWon%2Forchardgrid-app/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BingoWon%2Forchardgrid-app/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BingoWon%2Forchardgrid-app/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BingoWon","download_url":"https://codeload.github.com/BingoWon/orchardgrid-app/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BingoWon%2Forchardgrid-app/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278386651,"owners_count":25978217,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-04T02:00:05.491Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api-server","apple-intelligence","foundation-models","json-schema","macos","openai-api","structured-output","swift","swiftui"],"created_at":"2025-10-04T23:21:02.085Z","updated_at":"2026-04-15T04:02:12.276Z","avatar_url":"https://github.com/BingoWon.png","language":"Swift","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://orchardgrid.com/logo-with-text.svg\" alt=\"OrchardGrid\" height=\"80\" /\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cstrong\u003eApple Intelligence — from every device, for every app.\u003c/strong\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  A distributed compute pool of Apple devices, six on-device capabilities,\u003cbr/\u003e\n  one OpenAI-compatible API. Menu-bar app, CLI, or HTTP — your choice.\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/platform-iOS%20·%20iPadOS%20·%20macOS-blue?style=flat\" alt=\"Platform\" /\u003e\n  \u003cimg src=\"https://img.shields.io/badge/Swift-6-F05138?style=flat\u0026logo=swift\u0026logoColor=white\" alt=\"Swift 6\" /\u003e\n  \u003cimg src=\"https://img.shields.io/badge/macOS-26%2B-black?style=flat\u0026logo=apple\u0026logoColor=white\" alt=\"macOS 26+\" /\u003e\n  \u003cimg src=\"https://img.shields.io/badge/inference-100%25%20on--device-22c55e?style=flat\" alt=\"100% on-device\" /\u003e\n  \u003cimg src=\"https://img.shields.io/badge/MCP-supported-0066cc?style=flat\" alt=\"MCP supported\" /\u003e\n  \u003cimg src=\"https://img.shields.io/badge/license-BSL_1.1-green?style=flat\" alt=\"License\" /\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://apps.apple.com/us/app/orchardgrid/id6754092757\"\u003e\u003cstrong\u003eApp Store\u003c/strong\u003e\u003c/a\u003e ·\n  \u003ca href=\"https://orchardgrid.com\"\u003e\u003cstrong\u003eWebsite\u003c/strong\u003e\u003c/a\u003e ·\n  \u003ca href=\"https://orchardgrid.com/docs\"\u003e\u003cstrong\u003eAPI Docs\u003c/strong\u003e\u003c/a\u003e ·\n  \u003ca href=\"https://orchardgrid.com/dashboard\"\u003e\u003cstrong\u003eDashboard\u003c/strong\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"README.md\"\u003eEnglish\u003c/a\u003e · \u003ca href=\"README.zh-CN.md\"\u003e中文\u003c/a\u003e\n\u003c/p\u003e\n\n---\n\nApple Intelligence runs **only on Apple's Neural Engine** — it cannot be shipped to traditional cloud GPUs. OrchardGrid turns that constraint into a feature: install the app on any Mac, iPhone, or iPad; your device becomes a node in a **programmable pool of on-device AI**, reachable over your LAN or through OrchardGrid's cloud relay. Every byte of inference happens on the devices you own.\n\n## Screenshots\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/screenshots/ios-devices.png\" width=\"200\" alt=\"Devices on iOS\" /\u003e\n  \u0026nbsp;\u0026nbsp;\n  \u003cimg src=\"docs/screenshots/ios-device-detail.png\" width=\"200\" alt=\"Device detail\" /\u003e\n  \u0026nbsp;\u0026nbsp;\n  \u003cimg src=\"docs/screenshots/ios-api-keys.png\" width=\"200\" alt=\"API keys\" /\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/screenshots/macos-devices.png\" width=\"720\" alt=\"OrchardGrid on macOS\" /\u003e\n\u003c/p\u003e\n\n## ✨ Why OrchardGrid?\n\nOn-device AI projects usually pick one trade-off. OrchardGrid picks none of them.\n\n| | Cloud LLM APIs | [Ollama](https://ollama.com) | [apfel](https://github.com/Arthur-Ficial/apfel) | **OrchardGrid** |\n|---|:---:|:---:|:---:|:---:|\n| On-device inference | ❌ | ✅ | ✅ | ✅ |\n| Uses Apple's Neural Engine / foundation model | ❌ | ❌ | ✅ | ✅ |\n| Capabilities beyond chat (image · vision · speech · sound · NLP) | varies | ❌ | ❌ | **✅ all six** |\n| iOS and iPadOS | browser only | ❌ | ❌ | ✅ |\n| Menu-bar app (not just a CLI) | — | — | — | ✅ |\n| Pool many devices as one API | ❌ | ❌ | ❌ | ✅ |\n| Reachable from anywhere (your phone, CI, a teammate) | ✅ | localhost only | localhost only | ✅ |\n| OpenAI-compatible `/v1/*` | ✅ | ✅ | ✅ | ✅ |\n| MCP tool calling | varies | ❌ | ✅ | ✅ |\n| Free | ❌ | ✅ | ✅ | ✅ |\n\n**The short version.** If you have one Mac and want a single-binary CLI for Apple Intelligence, `apfel` is excellent — use it. If you want the same model **also from your iPhone**, **also via Image / Vision / Speech / Sound / NLP**, **also sharable with your team as a unified API**, **also packaged as a menu-bar app your family members can install from the App Store** — that's OrchardGrid.\n\n## 🚀 Quick Start\n\n### 1 · Install\n\n**Homebrew (macOS)** — one command, installs the app **and** the `og` CLI:\n\n```bash\nbrew install --cask bingowon/orchardgrid/orchardgrid\n```\n\nThe cask symlinks `/opt/homebrew/bin/og` into `OrchardGrid.app/Contents/Resources/og`. App and CLI share runtime state via the macOS App Group `group.com.orchardgrid.shared`.\n\n**App Store (iOS · iPadOS · macOS)**:\n\n\u003ca href=\"https://apps.apple.com/us/app/orchardgrid/id6754092757\"\u003e\n  \u003cimg src=\"https://tools.applemediaservices.com/api/badges/download-on-the-app-store/black/en-us?size=250x83\" alt=\"Download on the App Store\" height=\"42\" /\u003e\n\u003c/a\u003e\n\n**From source (macOS, development)** — clone, open `orchardgrid-app.xcodeproj`, build. Requires Xcode 26+.\n\n### 2 · First prompt\n\n```bash\nog \"What is the capital of Austria?\"\n```\n\nThat's it — the prompt runs **in-process** against `SystemLanguageModel.default`, with no local HTTP hop and no cloud round-trip.\n\n### 3 · Share or consume — pick any combination\n\n| You want to… | Do this |\n|---|---|\n| Use Apple Intelligence from the shell | `og \"...\"` (or `og --chat`) |\n| Call it from another app via OpenAI SDK | Enable **Share Locally** → hit `http://\u003cmac\u003e.local:8888/v1/chat/completions` |\n| Reach your Mac's AI from your iPhone / CI / a laptop | Enable **Share to Cloud** → `https://orchardgrid.com/v1/chat/completions` with your API key |\n| Do image / vision / speech / sound / NLP | Hit the corresponding `/v1/*` endpoint (see [Capabilities](#-capabilities)) |\n\n## ⌨️ The `og` CLI\n\n`og` is a thin, opinionated shell wrapper around Apple Intelligence. By default it runs on-device in the same process. Pass `--host` to talk to a peer or to OrchardGrid cloud instead.\n\n### Inference\n\n```bash\nog \"prompt\"                              # single-shot, streamed to stdout\nog --chat                                # interactive REPL, Ctrl-C to quit\nog --model-info                          # model availability, source, context size\nog -s \"You are a pirate.\" \"explain TCP\"  # system prompt\nog --system-file persona.txt \"...\"       # system prompt from file\nog --permissive \"creative writing...\"    # relax safety guardrails\n```\n\n### Files and stdin\n\n```bash\nog -f README.md \"summarise this\"                    # attach a file\nog -f old.swift -f new.swift \"what changed?\"        # multiple files\ngit diff HEAD~1 | og \"review this diff\"             # pipe stdin\ncat notes.txt | og -f extra.md \"merge and distil\"   # stdin + files mixed\n```\n\n### Output\n\n```bash\nog -o json \"one word capital of France\" | jq .content\nog --quiet \"one word capital of France\"             # no chrome, just the answer\nog --no-color \"...\"                                 # disable ANSI colour\n```\n\n### Generation options\n\n```bash\nog --temperature 0.2 \"...\"                          # deterministic-ish sampling\nog --max-tokens 100 \"...\"                           # cap completion length\nog --seed 42 \"...\"                                  # reproducible runs\n```\n\n### Context strategy — five ways to trim long conversations\n\n```bash\nog --chat --context-strategy newest-first    # default: keep the most recent turns\nog --chat --context-strategy oldest-first    # keep the earliest turns\nog --chat --context-strategy sliding-window --context-max-turns 20\nog --chat --context-strategy summarize       # compress old turns via a side model call\nog --chat --context-strategy strict          # refuse to trim — throw on overflow\n```\n\n### Tool calling via MCP (Model Context Protocol)\n\nAttach any stdio MCP server; `og` discovers its tools, registers them natively with `LanguageModelSession`, and Apple Intelligence decides when to call them.\n\n```bash\nog --mcp ./server.py \"what is 41 + 1?\"              # on-device tool call\nog --mcp ./a.py --mcp ./b.py --chat                 # multiple servers\nog --mcp-timeout 30 --mcp ./slow.py \"...\"           # per-call timeout\nog mcp list ./server.py                             # introspect a server's tools\nog mcp list ./server.py -o json                     # same, as JSON\n```\n\nMCP requires on-device inference — combining `--mcp` with `--host` is rejected at parse time.\n\n### Benchmark\n\n```bash\nog benchmark                                        # 5 runs against the local model\nog benchmark --runs 20 --bench-prompt \"Tell me a joke\"\nog benchmark --host http://mac.local:8888           # benchmark a peer\nog benchmark -o json --quiet | jq .tokensPerSec     # scriptable\n```\n\nReports **min / median / p95 / max / mean** for time-to-first-token, total latency, tokens/sec, and output tokens. Respects `--temperature` and `--max-tokens`.\n\n### Cloud account (after `og login`)\n\n```bash\nog login                            # OAuth loopback; opens browser, issues a management key\nog logout                           # drop local creds\nog logout --revoke                  # also revoke the key server-side\n\nog me                               # account info\nog keys                             # list API keys\nog keys create --name \"my-bot\"      # new inference-scope key, printed once\nog keys delete \u003chint\u003e               # revoke an inference key\nog devices                          # list your devices\nog logs --role self --limit 10      # recent usage\nog logs --role consumer --status failed --offset 20\n```\n\n### Remote endpoints\n\n```bash\nog --host https://orchardgrid.com --token sk-… \"hi\"   # cloud\nog --host http://mac.local:8888 \"hi\"                   # LAN peer\nORCHARDGRID_HOST=https://orchardgrid.com og \"hi\"       # via env\n```\n\n### Diagnostics\n\n```bash\nog status                # local server state, sharing toggles, login state\nog --version             # version\nog --help                # full flag reference\n```\n\nDeep-dive docs: [`docs/cli-reference.md`](docs/cli-reference.md) · [`docs/openai-api-compatibility.md`](docs/openai-api-compatibility.md) · [`docs/context-strategies.md`](docs/context-strategies.md) · [`demo/`](demo/) (capability-combining shell scripts).\n\n### Environment variables\n\n| Variable | Meaning |\n|---|---|\n| `ORCHARDGRID_HOST` | Default remote host (if unset, CLI runs on-device) |\n| `ORCHARDGRID_TOKEN` | Default bearer token |\n| `OG_NO_BROWSER` | Suppress `og login`'s auto browser launch (for SSH / CI) |\n| `NO_COLOR` | Disable ANSI colour output |\n\n### Exit codes\n\n| Code | Meaning |\n|:---:|---|\n| `0` | Success |\n| `1` | Runtime error (network, auth, unreachable) |\n| `2` | Usage error (bad flag, conflicting options) |\n| `3` | Guardrail blocked |\n| `4` | Context overflow |\n| `5` | Model unavailable (Apple Intelligence not enabled) |\n| `6` | Rate limited |\n\n## 🧠 Capabilities\n\nSix on-device Apple frameworks, one consistent OpenAI-flavoured interface. Every capability is reachable through the local API (`:8888` on your LAN) and the cloud relay (`https://orchardgrid.com`).\n\n| Capability | Framework | Endpoint | What it does |\n|---|---|---|---|\n| **Chat** | FoundationModels | `/v1/chat/completions` | LLM text generation, streaming, structured output, MCP tools |\n| **Image** | ImagePlayground | `/v1/images/generations` | Text-to-image (illustration and sketch styles) |\n| **NLP** | NaturalLanguage | `/v1/nlp/analyze` | Language detection, NER, tokenisation, embeddings |\n| **Vision** | Vision | `/v1/vision/analyze` | OCR, classification, face and barcode detection |\n| **Speech** | Speech | `/v1/audio/transcriptions` | Speech-to-text, 50+ languages |\n| **Sound** | SoundAnalysis | `/v1/audio/classify` | Environmental sound classification, ~300 categories |\n\n## 🌐 API access\n\n### Local (same LAN)\n\nEnable **Share Locally** in the app. The device listens on `:8888`:\n\n```bash\ncurl http://\u003cmac\u003e.local:8888/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"model\":\"apple-foundationmodel\",\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}]}'\n```\n\n```python\nfrom openai import OpenAI\nclient = OpenAI(base_url=\"http://mac.local:8888/v1\", api_key=\"unused\")\nclient.chat.completions.create(\n    model=\"apple-foundationmodel\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello\"}],\n)\n```\n\n### Cloud (reach your devices from anywhere)\n\nEnable **Share to Cloud** in the app, sign in, create an API key at [orchardgrid.com/dashboard/api-keys](https://orchardgrid.com/dashboard/api-keys):\n\n```bash\ncurl https://orchardgrid.com/v1/chat/completions \\\n  -H \"Authorization: Bearer sk-…\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"model\":\"apple-foundationmodel\",\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}]}'\n```\n\nThe cloud **never sees your prompt or response** — it routes a task id to an online device with the right capability, forwards the streamed bytes through, and logs the usage counters. Zero content storage by design.\n\n## 🏗 Architecture\n\n```mermaid\nflowchart LR\n    Client([Any OpenAI-compatible client])\n\n    subgraph Cloud[\"OrchardGrid Cloud · Cloudflare Workers\"]\n        direction TB\n        Auth[Auth \u0026amp; API keys]\n        Pool[\"DevicePoolManager\u003cbr/\u003eDurable Object\"]\n        DB[(D1 · SQLite)]\n        Auth --\u003e Pool\n        Pool \u003c--\u003e DB\n    end\n\n    subgraph Device[\"Your Apple devices · macOS · iOS · iPadOS\"]\n        direction TB\n        WS[WebSocket worker]\n        API[Local API server\u003cbr/\u003e:8888]\n        Cap[\"Chat · Image · NLP\u003cbr/\u003eVision · Speech · Sound\"]\n        WS --\u003e Cap\n        API --\u003e Cap\n    end\n\n    Client --\u003e|HTTPS · SSE| Auth\n    Pool \u003c--\u003e|WebSocket| WS\n    Client -.-\u003e|Direct LAN| API\n```\n\n**Reverse inference.** Unlike traditional AI services where the server owns the GPU, OrchardGrid's cloud has **zero compute** — it's a task router. Your devices sit behind NATs and firewalls; the server pushes tasks out over WebSocket and pipes results back. External clients see a plain HTTP + SSE API. Internally, every token is generated on hardware you own.\n\n## 🔒 Privacy\n\n- **100% on-device inference.** Apple's Neural Engine runs the model. Nothing leaves your device except the answer.\n- **Zero content storage in the cloud.** The relay forwards bytes and counts tokens — it does not persist prompts, completions, images, or audio.\n- **No telemetry.** No analytics, no crash-reporter uploading your queries.\n- **Audit the code.** The app, the CLI, the worker, and the database schema are all in this organisation's repos.\n- **Uninstall wipes state.** `brew uninstall --cask --zap` removes the App Group container and `~/.config/orchardgrid`.\n\n## 📊 Honest limits\n\n| Constraint | Detail |\n|---|---|\n| Context window | 4096 tokens (Apple Intelligence hard limit) |\n| Platform | Apple Silicon Macs (M1+) and Apple Intelligence-capable iPhone / iPad |\n| Model | One model per modality — whatever Apple ships |\n| Streaming | Chat and vision stream; image generation is one-shot |\n| Guardrails | Apple's safety system may refuse benign prompts; `--permissive` helps creative tasks |\n| Latency | On-device — single-digit seconds per response, no rate-limit but also no cloud GPU scale |\n| MCP | Stdio transport only; remote HTTP MCP servers are on the roadmap |\n\n## 🛠 Tech stack\n\n| Layer | Technology |\n|---|---|\n| Language | Swift 6 · strict concurrency · `@MainActor` managers |\n| UI | SwiftUI · macOS menu-bar + iOS navigation |\n| Networking | Apple Network framework (`NWListener`) · URLSession · WebSocket |\n| AI | FoundationModels · ImagePlayground · NaturalLanguage · Vision · Speech · SoundAnalysis |\n| Cloud backend | Cloudflare Workers · Durable Objects · D1 (SQLite) · Hono |\n| Auth | Clerk · Apple Sign-In · Bearer API keys (scoped: inference / management) |\n| Distribution | Homebrew cask (app + CLI) · App Store (iOS / iPadOS / macOS) |\n| CLI | Swift Package · 127 Swift Testing unit tests + 96 pytest integration tests |\n| Quality gate | GitHub Actions · CLI + Xcode app run on every push and PR |\n\n## 🤝 Contributing\n\nBug reports, feature ideas, pull requests — all welcome.\n\n1. Fork and branch from `main`.\n2. Commit in [Conventional Commits](https://www.conventionalcommits.org) style (`feat:` / `fix:` / `perf:` / `refactor:`) — these drive the automated release pipeline.\n3. Run `make format` (Swift) and `make test` before opening the PR.\n4. Describe the **why**, not just the what.\n\nNew to the codebase? Start with [CLAUDE.md](CLAUDE.md) — the operator guide — then grep the repo.\n\n\u003cp align=\"center\"\u003e\n  \u003csub\u003eBuilt with Swift 6 and Apple Silicon. No cloud GPUs were harmed in the making of this AI.\u003c/sub\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://orchardgrid.com\"\u003eWebsite\u003c/a\u003e \u0026nbsp;·\u0026nbsp;\n  \u003ca href=\"https://orchardgrid.com/docs\"\u003eAPI Docs\u003c/a\u003e \u0026nbsp;·\u0026nbsp;\n  \u003ca href=\"https://apps.apple.com/us/app/orchardgrid/id6754092757\"\u003eApp Store\u003c/a\u003e \u0026nbsp;·\u0026nbsp;\n  \u003ca href=\"https://orchardgrid.com/dashboard\"\u003eDashboard\u003c/a\u003e \u0026nbsp;·\u0026nbsp;\n  \u003ca href=\"orchardgrid-cli/README.md\"\u003eCLI reference\u003c/a\u003e\n\u003c/p\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FBingoWon%2Forchardgrid-apple","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FBingoWon%2Forchardgrid-apple","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FBingoWon%2Forchardgrid-apple/lists"}