{"id":51501113,"url":"https://github.com/quackscience/duckdb-mlx","last_synced_at":"2026-07-07T19:01:05.457Z","repository":{"id":369033008,"uuid":"1286582713","full_name":"quackscience/duckdb-mlx","owner":"quackscience","description":null,"archived":false,"fork":false,"pushed_at":"2026-07-03T09:14:18.000Z","size":203,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-07-03T10:14:09.335Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/quackscience.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-07-01T23:33:34.000Z","updated_at":"2026-07-03T09:14:22.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/quackscience/duckdb-mlx","commit_stats":null,"previous_names":["quackscience/duckdb-mlx"],"tags_count":null,"template":false,"template_full_name":"duckdb/extension-template","purl":"pkg:github/quackscience/duckdb-mlx","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quackscience%2Fduckdb-mlx","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quackscience%2Fduckdb-mlx/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quackscience%2Fduckdb-mlx/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quackscience%2Fduckdb-mlx/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/quackscience","download_url":"https://codeload.github.com/quackscience/duckdb-mlx/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quackscience%2Fduckdb-mlx/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35239467,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-07T02:00:07.222Z","response_time":90,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-07-07T19:01:04.368Z","updated_at":"2026-07-07T19:01:05.445Z","avatar_url":"https://github.com/quackscience.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# duckdb-mlx\n\nGPU-accelerated DuckDB on Apple Silicon (Metal / MLX).\n\nThe loadable extension is named **`mlx`** (`LOAD mlx;`). See [docs/PLAN.md](docs/PLAN.md) for\narchitecture and roadmap. Progress is test-gated: every capability lands with a passing\ndifferential test against DuckDB's CPU engine before it is committed.\n\n**Status (2026-07-03):** Transparent GPU acceleration is live. Optimizer intercepts supported\nplans with **shape-aware decline** (CPU keeps scan-bound and cold low-card GROUP BY). GPU-resident\ncolumn cache serves repeated queries without rescanning. **Slot-lock + LSD radix GROUP BY**\non Metal (32K partitions, hybrid dispatch) with `SET mlx_groupby_path = 'dense'|'slotlock'|'radix'|'sort'|'auto'`.\nTPC-H Q1 uses a **pin-time bundle + dedicated/fused Metal kernels** (see `tpch_q1_pinned.test`).\n\n## Quick benchmark (repeatable)\n\n```shell\nGEN=ninja make release\n./build/release/test/unittest \"[sql]\"          # differential SQL suite (see test/sql/)\n\nbenchmark/run_all.sh 1                         # minimal + roofline + Q1 @ SF1\nbenchmark/bench_roofline.sh 1                  # stream SUM + multi-agg GiB/s\nbenchmark/bench_q1.sh 1                        # official TPC-H Q1 CPU vs GPU (pinned)\nbenchmark/bench_groupby_paths.sh               # GROUP BY path microbench (small scale)\npython3 benchmark/tpch/run.py 1                # full 22-query GQE harness\n```\n\n**GROUP BY path testing (local → M3 Ultra):**\n\n```sql\nSET mlx_groupby_path = 'slotlock';  -- or dense | radix | sort | auto\nSELECT mlx_groupby_bench(list(g), list(v::BIGINT)) FROM t;\n```\n\nSee `test/sql/mlx_groupby_paths.test`, `mlx_groupby_stability.test`, `mlx_decline.test`.\n\n**SF10:** use `bench_roofline.sh 10` and `bench_q1.sh 10` with **lineitem-only pin** inside\nthose scripts. Full `mlx_cache_pin_tpch()` on SF10 can hit Metal memory limits on 24 GB machines.\n\nDebug Q1 fast path: `SET mlx_log_level=debug;` — look for `MLX Q1 fast path: fused grid-stride`.\n\nMetal kernels use **grid-stride reductions** (capped threadgroups), **partitioned slot-lock\nhash aggregates** (no 64-bit device atomics on Apple GPUs), and **LSD radix sort** with\non-device bucket scan for high-cardinality GROUP BY.\n\n## Headline: plain SQL, 14× (base M4, 100M rows, hot cache)\n\n```sql\nLOAD mlx;\nSELECT sum(sin(x) * cos(x) + sqrt(abs(x) + 1)) FROM t;  -- plain SQL, no special syntax\n```\n\n| Query (100M rows) | DuckDB CPU (all cores) | duckdb-mlx GPU (hot) | Speedup |\n|---|---|---|---|\n| `sum(sin(x)*cos(x)+sqrt(abs(x)+1))` | 190 ms | **13 ms** | **14×** |\n| `sum(x)` | 13 ms | **4 ms** | 3.3× (fp32 bandwidth limit) |\n\nFirst run over a table is cold (~0.5 s: scan + GPU cache build), then the columns stay\nGPU-resident; row-count changes invalidate and repopulate automatically. During hot GPU\nqueries the CPU does ~1 ms of work versus 1.6 CPU-seconds on the CPU engine. Every result\nis verified against DuckDB's own engine by the differential test suite. Reproduce with\n`benchmark/bench_transparent.sql`.\n\n**Transparent coverage so far:** ungrouped `sum` / `count` / `count(*)` / `avg` / `min` /\n`max` over expressions of `+ - * /`, `sin`, `cos`, `sqrt`, `abs`, casts and constants on\nDOUBLE/FLOAT/BIGINT/INTEGER columns — including **WHERE clauses**, which execute as GPU\nmasks over the resident columns (both pushed-down comparisons and residual predicates\nwith `AND`/`OR`/`NOT`). NULL semantics are honored throughout; anything unsupported\ndeclines silently to DuckDB's CPU engine, and correctness always wins over speed\n(the GPU pipeline computes in fp32 — counts are exact, floating aggregates agree with\nthe CPU engine to ~1e-7 relative in testing).\n\n## Flagship: GPU-resident vector search\n\n```sql\nLOAD mlx;\n\n-- one-time: pin an embedding column as a GPU-resident, L2-normalized matrix\nSELECT mlx_vss_pin('items', list(emb ORDER BY id)) FROM items;\n\n-- cosine top-k, one matvec against the pinned matrix\nSELECT idx, score FROM mlx_vss_search('items', [0.12, 0.34, ...], 10);\n\n-- Q queries in a single matmul (reranking / batch scoring / similarity joins)\nSELECT query_no, idx, score FROM mlx_vss_search_batch('items', [[...], [...]], 10);\n```\n\nMeasured on a base M4 (24 GB), 1M vectors × 384 dims fp32 (~1.5 GB), hot cache,\nverified result-identical to DuckDB's `array_cosine_similarity` brute force:\n\n| Workload | DuckDB CPU (all cores) | duckdb-mlx GPU | Speedup |\n|---|---|---|---|\n| 1 query, top-10 | 70 ms | 20 ms | 3.5× (at bandwidth floor) |\n| 32 queries | ~1.7 s | ~105 ms | **16×** |\n| 128 queries | ~6.5 s | ~375 ms | **17×** |\n\nPin cost is ~5 s one-time and amortizes across all queries. Reproduce with\n`benchmark/bench_vss.sql` and `benchmark/bench_vss_batch.sql`.\n\n## All functions\n\n| Function | Purpose |\n|---|---|\n| `mlx_info()` | extension/GPU/MLX/spdlog status string |\n| `mlx_selftest()` | end-to-end GPU sanity check, returns `ok` |\n| `mlx_cache_pin(table)` / `mlx_cache_pin_tpch()` | GPU-resident column cache |\n| `mlx_stream_sum_bench(col_key)` | roofline SUM on pinned column (`lineitem#5`) |\n| `mlx_multi_agg_bench(col_key)` | roofline SUM+MIN+MAX+COUNT fusion on pinned column |\n| `mlx_vss_pin(name, list(col))` | pin a `FLOAT[N]` column as a GPU-resident matrix |\n| `mlx_vss_search(name, query, k)` | cosine top-k against a pinned matrix |\n| `mlx_vss_search_batch(name, queries, k)` | batched top-k, one matmul |\n| `mlx_sum(BIGINT[])`, `mlx_expr_bench(BIGINT[])` | Phase 0 spike/benchmark vehicles |\n\nSettings: `mlx_execution` (opt-in GPU plans), `mlx_min_rows`, `mlx_log_level`.\n\nThe extension builds on all DuckDB platforms; the GPU path is compiled in only on Apple\nSilicon (`DUCKDB_MLX_ENABLE_GPU`, auto-detected). CI distribution builds **osx_arm64 only**.\n\n## Repository layout\n\n```\nsrc/\n├── duckdb_mlx_extension.cpp   # load/register, settings, spike functions\n├── ops/mlx_vss.cpp            # vector-search scalar + table functions\n├── bridge/mlx_bridge.cpp      # the only TU that includes MLX headers\n├── mlx_logger.cpp             # the only TU that includes spdlog headers\n├── planner/ format/ kernels/ exec/   # reserved for Phase 1+ (see docs/PLAN.md §5)\n└── include/                   # bridge/logger interfaces (no MLX/spdlog types)\ntest/sql/                      # sqllogictests incl. GPU-vs-CPU differentials\nbenchmark/                     # reproduction scripts for the numbers above\nthird_party/mlx/               # MLX v0.31.2 submodule (Apple-only, not vcpkg)\n```\n\nIsolation rule (learned the hard way): DuckDB's vendored fmt shadows the real fmt for\nevery extension target, so MLX and spdlog each live in a dedicated static library whose\ninherited include paths are cleared — no translation unit ever includes both `duckdb.hpp`\nand MLX/spdlog headers. Data crosses the bridge as raw unified-memory pointers.\n\n## Building\n\nRequires **macOS 14+**, **full Xcode.app** (Apple's IDE — not Command Line Tools alone),\nCMake, and ninja. MLX compiles Metal shaders during the build.\n\n**There is no Homebrew formula for the Metal compiler.** `brew install cmake ninja`\nis fine; **Xcode itself must come from Apple** (App Store or developer download).\n\nCheck the toolchain before `make`:\n\n```shell\n./scripts/check_toolchain.sh\n```\n\nYou need `xcrun -sdk macosx metal --version` to succeed.\n\n### Install Xcode (first time — no Xcode on disk yet)\n\nUse **one** of these. Do **not** use `brew install xcodesorg/made/xcodes` for the\nfirst install — that Homebrew formula fails without Xcode already present (`xcbuild`\nmissing).\n\n1. **App Store** — search “Xcode”, Install, open once to finish setup.\n\n2. **Terminal via `mas`** (optional):\n\n```shell\nbrew install mas\nmas install 497799835\nopen -a Xcode\n```\n\n3. **[Apple Developer downloads](https://developer.apple.com/download/all/)** — download\n   the `.xip`, extract, drag `Xcode.app` to `/Applications`, open once.\n\nThen activate the toolchain:\n\n```shell\nsudo xcode-select -s /Applications/Xcode.app/Contents/Developer\nsudo xcodebuild -license accept\nxcrun -sdk macosx metal --version\n```\n\nIf Xcode is named `Xcode-16.x.app`, run `./scripts/check_toolchain.sh` for the exact\n`xcode-select` path.\n\nThe [`xcodes`](https://github.com/XcodesOrg/xcodes) CLI is only useful **after** you\nalready have one Xcode — for installing/switching versions side by side.\n\nPortable C++ dependencies (currently **spdlog**) are declared in `vcpkg.json` and\ninstalled automatically on first build — `make` bootstraps a local `vcpkg/` checkout\nif needed. MLX is vendored as a submodule and built into `build/mlx-install` on first\nconfigure (`scripts/build_mlx.sh`, Metal-only — no JACCL/CPU backend).\n\n```shell\ngit submodule update --init --recursive\nGEN=ninja make release\n```\n\nIf an earlier `make release` failed before vcpkg was set up, clear the stale CMake\ncache and rebuild:\n\n```shell\nrm -rf build/release\nGEN=ninja make release\n```\n\nIf the vendored MLX build failed (e.g. after an Xcode/SDK upgrade), clear the MLX\nartifacts and retry:\n\n```shell\nrm -rf build/mlx-build build/mlx-install\nGEN=ninja make release\n```\n\n(`make release` also detects and removes poisoned caches automatically.)\n\nTo use a system-wide vcpkg install instead, set `VCPKG_TOOLCHAIN_PATH` before\n`make` (see [DuckDB extension vcpkg docs](duckdb/extension/README.md)).\n\nMain artifacts:\n\n```shell\n./build/release/duckdb                                            # shell, extension pre-loaded\n./build/release/test/unittest                                     # test runner\n./build/release/extension/mlx/mlx.duckdb_extension  # loadable extension\n```\n\n## Testing\n\n```shell\nGEN=ninja make test\n# or:\n./build/release/test/unittest \"[sql]\"\n```\n\nSQL logic tests live in `test/sql/` (`require mlx;`). Every GPU capability is diffed against\nDuckDB's CPU results.\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fquackscience%2Fduckdb-mlx","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fquackscience%2Fduckdb-mlx","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fquackscience%2Fduckdb-mlx/lists"}