{"id":51008241,"url":"https://github.com/john-rocky/coreai-model-zoo","last_synced_at":"2026-06-20T23:04:15.822Z","repository":{"id":363739115,"uuid":"1264692184","full_name":"john-rocky/coreai-model-zoo","owner":"john-rocky","description":"Community model zoo + knowledge base for Apple Core AI (iOS/macOS 27): Qwen3.5 \u0026 Gemma 4 converted end-to-end, verified on-device (iPhone 17 Pro GPU/ANE), conversion gotchas, custom Metal kernels, Swift runner","archived":false,"fork":false,"pushed_at":"2026-06-19T08:03:20.000Z","size":642,"stargazers_count":163,"open_issues_count":1,"forks_count":11,"subscribers_count":4,"default_branch":"main","last_synced_at":"2026-06-19T09:29:01.254Z","etag":null,"topics":["ai","apple-silicon","coreai","coreml","gemma4","granite","ios","iphone","lfm","llm","local-llm","macos","minicpm","qwen"],"latest_commit_sha":null,"homepage":"","language":"Swift","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/john-rocky.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-10T05:21:10.000Z","updated_at":"2026-06-19T08:03:24.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/john-rocky/coreai-model-zoo","commit_stats":null,"previous_names":["john-rocky/coreai-model-zoo"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/john-rocky/coreai-model-zoo","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/john-rocky%2Fcoreai-model-zoo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/john-rocky%2Fcoreai-model-zoo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/john-rocky%2Fcoreai-model-zoo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/john-rocky%2Fcoreai-model-zoo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/john-rocky","download_url":"https://codeload.github.com/john-rocky/coreai-model-zoo/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/john-rocky%2Fcoreai-model-zoo/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34588006,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-20T02:00:06.407Z","response_time":98,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","apple-silicon","coreai","coreml","gemma4","granite","ios","iphone","lfm","llm","local-llm","macos","minicpm","qwen"],"created_at":"2026-06-20T23:04:15.323Z","updated_at":"2026-06-20T23:04:15.814Z","avatar_url":"https://github.com/john-rocky.png","language":"Swift","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CoreAI-Model-Zoo\n\nLLMs converted to Apple **Core AI** (`.aimodel`, iOS 27 / macOS 27) — downloadable, verified\non-device, with the conversion code and a knowledge base. Successor to\n[`CoreML-Models`](https://github.com/john-rocky/CoreML-Models).\n\n## Models\n\n| Model | Download (`.aimodel`) | License |\n|---|---|---|\n| **Qwen3.5-0.8B** | [🤗 qwen3.5-0.8B-CoreAI](https://huggingface.co/mlboydaisuke/qwen3.5-0.8B-CoreAI) | Apache-2.0 |\n| **Qwen3.5-2B** | [🤗 qwen3.5-2B-CoreAI](https://huggingface.co/mlboydaisuke/qwen3.5-2B-CoreAI) | Apache-2.0 |\n| **Qwen3.6-35B-A3B** (MoE, Mac-only) | [🤗 Qwen3.6-35B-A3B-CoreAI](https://huggingface.co/mlboydaisuke/Qwen3.6-35B-A3B-CoreAI) | Apache-2.0 |\n| **Qwen3.6-27B** (dense, Mac-only) | [🤗 Qwen3.6-27B-CoreAI](https://huggingface.co/mlboydaisuke/Qwen3.6-27B-CoreAI) | Apache-2.0 |\n| **GLM-4.7-Flash** (MoE + MLA, Mac-only — zoo's first MLA) | [🤗 GLM-4.7-Flash-CoreAI](https://huggingface.co/mlboydaisuke/GLM-4.7-Flash-CoreAI) | MIT |\n| **Gemma 4 E2B** (text, incl. official-QAT int4) | [🤗 gemma-4-E2B-CoreAI](https://huggingface.co/mlboydaisuke/gemma-4-E2B-CoreAI) | Gemma |\n| **Gemma 4 E4B** (text, official-QAT int4) | [🤗 gemma-4-E4B-CoreAI](https://huggingface.co/mlboydaisuke/gemma-4-E4B-CoreAI) | Gemma |\n| **Gemma 4 12B** (dense, Mac-only — custom flash-decode kernel ‡) | [🤗 Gemma-4-12B-CoreAI](https://huggingface.co/mlboydaisuke/Gemma-4-12B-CoreAI) | Gemma |\n| **Gemma 4 31B** (dense, Mac-only — custom flash-decode kernel ‡) | [🤗 Gemma-4-31B-CoreAI](https://huggingface.co/mlboydaisuke/Gemma-4-31B-CoreAI) | Gemma |\n| **LFM2.5-1.2B-Instruct** | [🤗 LFM2.5-1.2B-CoreAI](https://huggingface.co/mlboydaisuke/LFM2.5-1.2B-CoreAI) | LFM Open License v1.0 |\n| **LFM2.5-8B-A1B** (MoE, custom `gather_qmm` kernel — first iPhone MoE) | [🤗 LFM2.5-8B-A1B-CoreAI](https://huggingface.co/mlboydaisuke/LFM2.5-8B-A1B-CoreAI) | LFM Open License v1.0 |\n| **Granite 4.0-H 1B / 350M** | [🤗 granite-4.0-h-CoreAI](https://huggingface.co/mlboydaisuke/granite-4.0-h-CoreAI) | Apache-2.0 |\n| **Nanbeige4.1-3B** (dense reasoning/agentic, iPhone — 32B-class @ 3.93B) | [🤗 Nanbeige4.1-3B-CoreAI](https://huggingface.co/mlboydaisuke/Nanbeige4.1-3B-CoreAI) | Apache-2.0 |\n| **Qwen3-VL** (vision-language) | [🤗 2B](https://huggingface.co/mlboydaisuke/Qwen3-VL-2B-CoreAI) · [4B](https://huggingface.co/mlboydaisuke/Qwen3-VL-4B-CoreAI) · [8B](https://huggingface.co/mlboydaisuke/Qwen3-VL-8B-CoreAI) | Apache-2.0 |\n| **MiniCPM-V 4.6** (vision-language, sub-2B — strongest tiny VLM) | [🤗 MiniCPM-V-4.6-CoreAI](https://huggingface.co/mlboydaisuke/MiniCPM-V-4.6-CoreAI) | Apache-2.0 |\n| **Gemma 4 E2B vision (VL)** (image+text) | `vl/` in [🤗 gemma-4-E2B-CoreAI](https://huggingface.co/mlboydaisuke/gemma-4-E2B-CoreAI) | Gemma |\n| **Qwen2.5-Omni-3B Audio** (audio *understanding* — describes sounds, not a transcript; iPhone + Mac, zoo's first audio model) | [🤗 Qwen2.5-Omni-3B-Audio-CoreAI](https://huggingface.co/mlboydaisuke/Qwen2.5-Omni-3B-Audio-CoreAI) | Apache-2.0 |\n| **EmbeddingGemma 300M** (text embeddings — on-device RAG / semantic search) | [🤗 embeddinggemma-300m-CoreAI](https://huggingface.co/mlboydaisuke/embeddinggemma-300m-CoreAI) | Gemma |\n| **Qwen3-Embedding 0.6B** (multilingual text embeddings, last-token pooling + MRL) | [🤗 Qwen3-Embedding-0.6B-CoreAI](https://huggingface.co/mlboydaisuke/Qwen3-Embedding-0.6B-CoreAI) | Apache-2.0 |\n| **Qwen3-Reranker 0.6B** (cross-encoder reranker — yes/no relevance score) | [🤗 Qwen3-Reranker-0.6B-CoreAI](https://huggingface.co/mlboydaisuke/Qwen3-Reranker-0.6B-CoreAI) | Apache-2.0 |\n| **RF-DETR nano/small/medium/large** (object detection, no NMS) | [🤗 RF-DETR-CoreAI](https://huggingface.co/mlboydaisuke/RF-DETR-CoreAI) | Apache-2.0 |\n| **RF-DETR-Seg nano→2xlarge** (instance segmentation, 6 sizes) | [🤗 RF-DETR-CoreAI](https://huggingface.co/mlboydaisuke/RF-DETR-CoreAI) | Apache-2.0 |\n\n### Decode throughput (tok/s, greedy; output top-1 exact vs the Hugging Face reference)\n\n| | iPhone 17 Pro · GPU | iPhone 17 Pro · ANE | M4 Max · GPU |\n|---|---|---|---|\n| **Qwen3.5-0.8B** | **71.9** | 14.7 | **210** |\n| **Qwen3.5-2B** | **29** | — | **161** |\n| **LFM2.5-1.2B** | **45.4** | — | **276.5** |\n| **Granite 4.0-H 1B** | **36.3** | — | **136.5** |\n| **Nanbeige4.1-3B** | **15.9** | — | **114.5** |\n| **Gemma 4 E2B** | **30.3** (QAT 30.7) | 6 | **77.0** (QAT 78.9) |\n| **Gemma 4 E4B** (official QAT) | **15.1** | — | **55.8** |\n| **Gemma 4 E2B VL** (image+text, official QAT) | **25.5** | — | **82.4** |\n| **MiniCPM-V 4.6** (vision-language, sub-2B) | **53.4** | — | **224.3** |\n| **Qwen3.6-35B-A3B** (MoE, 35B/~3B active, Mac-only) | — | — | **64.9** † |\n| **Qwen3.6-27B** (dense, Mac-only) | — | — | **15.9** |\n| **GLM-4.7-Flash** (MoE + MLA, 30B/~3B active, Mac-only) | — | — | **52.4** † |\n| **Gemma 4 12B** (dense, Mac-only) | — | — | **23** int8 / **33** int4 ‡ |\n| **Gemma 4 31B** (dense, Mac-only) | — | — | **17.2** int4 ‡ |\n\nMeasured on the iOS 27 / macOS 27 beta, Apple's `coreai-pipelined` GPU engine, zero custom\nkernels (ANE column + **†**/**‡** excepted). **†** = MoE bundle using the custom\n[`gather_qmm`](knowledge/compute-units-and-authoring.md) Metal kernel (reads only the routed\nexperts). **‡** = dense bundle whose full/global-attention SDPA is a custom flash-decode Metal\nkernel — the stock MPSGraph SDPA crashes on the ≥16-head × 512 Q (a GPU scratch-heap overflow,\n[apple/coreai-models#27](https://github.com/apple/coreai-models/issues/27)), so these models are\n**unrunnable without it**. Prefill, sizes, per-model caveats, and the Mac-only big models: [`zoo/`](zoo/).\n\n\u003cp align=\"center\"\u003e\n  \u003cimg width=\"380\" alt=\"CoreAIChat screen recording\" src=\"https://github.com/user-attachments/assets/999dbd95-45b5-468f-b1a8-34112ee3b74d\" /\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\u003ci\u003eCoreAIChat (\u003ca href=\"apps/\"\u003eapps/\u003c/a\u003e) — the zoo's models running on-device on iPhone.\u003c/i\u003e\u003c/p\u003e\n\n## Start here\n\n- **Try the app** (iOS 27 / macOS 27 beta; the model downloads in-app):\n  - **Demo app, no build** → Mac: [**.dmg**](https://github.com/john-rocky/coreai-model-zoo/releases/download/mac-v1.0/CoreAI-Zoo-for-Mac.dmg) (notarized, runs the Mac-only bundles) · iPhone: CoreAIChat on TestFlight (coming soon)\n  - **Build it** → [`apps/`](apps/) — Xcode 27 beta + xcodegen, the `coreai-models` patch stack + `tokenizer.json`\n- **Run a model in your own app** → [`knowledge/swift-runtime.md`](knowledge/swift-runtime.md) + the model card\n- **Convert a model** → [`knowledge/conversion-guide.md`](knowledge/conversion-guide.md)\n- **Compress** → [`knowledge/compression.md`](knowledge/compression.md)\n- **Make it fast** → [`knowledge/custom-metal-kernels.md`](knowledge/custom-metal-kernels.md) · [`knowledge/performance-ceiling.md`](knowledge/performance-ceiling.md)\n- **Known beta issue** (in-graph KV-write crash; workarounds + the input-mask escape) → [`knowledge/coreai-beta-mpsgraph-kvwrite-bug.md`](knowledge/coreai-beta-mpsgraph-kvwrite-bug.md) — FB23024751 / [apple/coreai-models#5](https://github.com/apple/coreai-models/issues/5)\n\n## Repository layout\n\n| Dir | What |\n|---|---|\n| [`zoo/`](zoo/) | Model cards — configurations, sizes, parity, measured throughput. |\n| [`knowledge/`](knowledge/) | Verified notes on the framework: conversion, compression, stateful KV, custom Metal kernels, AOT, compute-unit rules, the Swift runtime. |\n| [`conversion/`](conversion/) | Re-authored models + convert / verify / compress scripts (PyTorch → `.aimodel`). |\n| [`swift/`](swift/) | `CoreAIRunner` — a Swift package that drives `.aimodel` LLM bundles, including architectures beyond the standard runtime. |\n| [`apps/`](apps/) | SwiftUI on-device chat apps (iOS 27): CoreAIChat (Gemma 4 E2B GPU/ANE/⚡ + Qwen3.5 / Qwen3.5-2B / LFM2.5 / Granite ⚡pipelined, one picker) + QwenChatFast (Qwen3.5 static kernels) with in-app model download. |\n\n## License\n\nBSD-3-Clause ([`LICENSE`](LICENSE)). Re-authored model code derives from Apple's BSD-3-Clause\n`coreai_models` and retains its notices. Model weights follow their own licenses (see each\nHugging Face repo).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjohn-rocky%2Fcoreai-model-zoo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjohn-rocky%2Fcoreai-model-zoo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjohn-rocky%2Fcoreai-model-zoo/lists"}