{"id":30767646,"url":"https://github.com/rikulauttia/ai-gpu-playground-mac","last_synced_at":"2026-05-06T03:32:10.372Z","repository":{"id":310408487,"uuid":"1039733075","full_name":"rikulauttia/ai-gpu-playground-mac","owner":"rikulauttia","description":"Hands-on CPU vs GPU benchmarks for Apple Silicon (M-series): PyTorch MPS, TensorFlow-Metal, MLX, and llama.cpp to measure TFLOP/s \u0026 tokens/sec and learn why GPUs accelerate training.","archived":false,"fork":false,"pushed_at":"2025-08-17T22:34:26.000Z","size":5,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-04T20:11:54.046Z","etag":null,"topics":["apple-silicon","benchmark","deep-learning","education","gpu","hands-on","llamacpp","matrix-multiplication","metal","mlx","mps","pytorch","tensorflow","tflops","tokens-per-second"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rikulauttia.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-08-17T21:44:02.000Z","updated_at":"2025-08-17T23:15:05.000Z","dependencies_parsed_at":"2025-08-17T23:30:54.449Z","dependency_job_id":"1520b24b-4997-4671-816c-c2f024df9a93","html_url":"https://github.com/rikulauttia/ai-gpu-playground-mac","commit_stats":null,"previous_names":["rikulauttia/ai-gpu-playground-mac"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/rikulauttia/ai-gpu-playground-mac","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rikulauttia%2Fai-gpu-playground-mac","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rikulauttia%2Fai-gpu-playground-mac/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rikulauttia%2Fai-gpu-playground-mac/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rikulauttia%2Fai-gpu-playground-mac/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rikulauttia","download_url":"https://codeload.github.com/rikulauttia/ai-gpu-playground-mac/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rikulauttia%2Fai-gpu-playground-mac/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32677926,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-06T02:33:58.958Z","status":"ssl_error","status_checked_at":"2026-05-06T02:33:39.611Z","response_time":117,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apple-silicon","benchmark","deep-learning","education","gpu","hands-on","llamacpp","matrix-multiplication","metal","mlx","mps","pytorch","tensorflow","tflops","tokens-per-second"],"created_at":"2025-09-04T20:07:10.221Z","updated_at":"2026-05-06T03:32:10.345Z","avatar_url":"https://github.com/rikulauttia.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AI GPU Playground (Mac, Apple Silicon)\n\n\u003e Touch the gradient. Watch matrices fly.  \n\u003e This tiny lab lets you _feel_ why GPUs matter: measure CPU vs Metal GPU on your Mac.\n\n![Platform](\u003chttps://img.shields.io/badge/platform-macOS%20(M1%2FM2%2FM3%2FM4)-informational\u003e)\n![PyTorch](https://img.shields.io/badge/PyTorch-MPS-blue)\n![TensorFlow](https://img.shields.io/badge/TensorFlow-Metal-orange)\n![MLX](https://img.shields.io/badge/Apple-MLX-success)\n![License](https://img.shields.io/badge/License-MIT-lightgrey)\n\nHands-on mini-project to _feel_ what \"more GPU\" means for AI training by comparing CPU vs Apple's GPU (via **Metal** / **MPS**) on your MacBook Pro (M-series).\n\n## What you'll do\n\n1. **PyTorch (MPS)** – measure big matrix multiply speed (the core of AI training) on `cpu` vs `mps`.\n2. **TensorFlow (Metal)** – the same idea with TensorFlow \u0026 `tensorflow-metal` plugin.\n3. **(Optional) llama.cpp** – run a small LLM locally and watch _tokens/sec_ jump when offloading layers to GPU.\n4. **(Optional) MLX** – Apple's own framework for Apple silicon. Try quick inference / fine-tuning on small models.\n\n\u003e Matrix multiply is the main workload in deep learning (GEMM). Faster GEMM ⇒ faster training.\n\n---\n\n## 0) Prereqs\n\n- macOS 13+ (Ventura or newer), Apple Silicon (M1/M2/M3/M4).\n- Python 3.10+ recommended.\n- Xcode command line tools: `xcode-select --install` (or open Xcode once).\n- **Tip:** Open **Activity Monitor → Window → GPU History** to _see_ GPU usage during runs.\n\n---\n\n## 1) PyTorch (MPS) – CPU vs GPU benchmark\n\n\u003e The **MPS** backend lets PyTorch use the Apple GPU through Metal.\n\n### Create an env \u0026 install\n\n```bash\npython3 -m venv .venv-torch\nsource .venv-torch/bin/activate\npython -m pip install --upgrade pip\npip install torch torchvision torchaudio   # ← no CPU-only index URL\npython - \u003c\u003c'PY'\nimport torch\nprint(\"torch:\", torch.__version__)\nprint(\"MPS available:\", torch.backends.mps.is_available())\nprint(\"MPS built:\", torch.backends.mps.is_built())\nPY\n```\n\n\u003e If `is_available()` is `True`, you’re good. If not, make sure macOS and Xcode tools are up to date.\n\n### Run the matmul benchmark\n\n```bash\n# 1) Apples-to-apples baseline (same size \u0026 dtype)\npython benchmarks/pytorch_mps_matmul.py --device cpu --size 4096 --repeat 3 --dtype float32\npython benchmarks/pytorch_mps_matmul.py --device mps  --size 4096 --repeat 3 --dtype float32\n\n# 2) Show GPU advantage with mixed precision\npython benchmarks/pytorch_mps_matmul.py --device mps  --size 4096 --repeat 5 --dtype float16\n```\n\nYou’ll see wall‑clock times and approximate TFLOP/s. The GPU (`mps`) should be significantly faster than `cpu` on larger sizes, especially with `float16`.\n\n---\n\n## 2) TensorFlow (Metal) – CPU vs GPU benchmark\n\n### Create an env \u0026 install\n\n```bash\npython3 -m venv .venv-tf\nsource .venv-tf/bin/activate\npython -m pip install --upgrade pip\npip install tensorflow tensorflow-metal\npython - \u003c\u003c'PY'\nimport tensorflow as tf\nprint(\"tf:\", tf.__version__)\nprint(\"GPU devices:\", tf.config.list_physical_devices(\"GPU\"))\nPY\n```\n\n### Run the matmul benchmark\n\n```bash\n# 1) Apples-to-apples baseline (same size \u0026 dtype)\npython benchmarks/tf_metal_matmul.py --device cpu --size 4096 --repeat 3 --dtype float32\npython benchmarks/tf_metal_matmul.py --device gpu --size 4096 --repeat 3 --dtype float32\n\n# 2) Show GPU advantage with mixed precision\npython benchmarks/tf_metal_matmul.py --device gpu --size 4096 --repeat 5 --dtype float16\n\n# (Optional) Heavier run\npython benchmarks/tf_metal_matmul.py --device cpu --size 8192 --repeat 1 --dtype float32\npython benchmarks/tf_metal_matmul.py --device gpu --size 8192 --repeat 3 --dtype float16\n```\n\nIf GPU is set up, you’ll see one Metal GPU in the device list and faster timings for `--device gpu` on larger sizes.\n\n---\n\n## 3) (Optional) llama.cpp – local LLM with Metal\n\nBuild llama.cpp with Metal and test tokens/sec.\n\n```bash\n# prerequisites: cmake, git, build tools (e.g. via Homebrew: brew install cmake)\ngit clone https://github.com/ggerganov/llama.cpp\ncd llama.cpp\nmake LLAMA_METAL=1\n# Download a small GGUF model (example: a 1–2B instruct model)\n# (Use the model vendor’s download link; place it under ./models)\n# Run with GPU offload of all layers (-ngl 99)\n./main -m models/YourSmallModel.gguf -p \"Say hello in Finnish.\" -ngl 99\n```\n\nCheck the printed **tokens per second**. Compare with `-ngl 0` (CPU only). Metal offload should increase throughput.\n\n---\n\n## 4) (Optional) MLX (Apple’s framework)\n\n```bash\npython3 -m venv .venv-mlx\nsource .venv-mlx/bin/activate\npip install mlx mlx-lm\n# Quick generation with a tiny MLX model\npython - \u003c\u003c'PY'\nfrom mlx_lm import load, generate\nmodel, tokenizer = load(\"mlx-community/Qwen2.5-0.5B-Instruct-mlx\")\nprint(generate(model, tokenizer, prompt=\"Miksi GPU nopeuttaa neuroverkkojen opetusta?\", max_tokens=64))\nPY\n```\n\nMLX is optimized for Apple silicon and can use the GPU/ANE under the hood.\n\n---\n\n## Why this demonstrates “many GPUs”\n\n- **Training = tons of matrix multiplies.** GPUs do thousands of these in parallel. CPU has few wide cores; GPU has many smaller cores.\n- **More (and bigger) GPUs ⇒ more throughput \u0026 memory.** You can use **data parallelism** (same model on many GPUs, split the batch) or **model/tensor parallelism** (split the model across GPUs) to scale. For huge LLMs, clusters of GPUs are linked with high‑speed interconnects.\n- On a Mac you won’t train giant LLMs from scratch, but you can **feel the acceleration** and do **fine‑tuning of small models**.\n\n---\n\n## Repo structure\n\n```\nbenchmarks/\n  pytorch_mps_matmul.py\n  tf_metal_matmul.py\nllama/\n  README.md\nREADME.md\n```\n\n---\n\n## Troubleshooting\n\n- If MPS/TensorFlow GPU isn’t detected, update macOS and Xcode CLTs, and ensure you’re in the correct virtualenv.\n- For PyTorch MPS, set `PYTORCH_ENABLE_MPS_FALLBACK=1` to fall back to CPU ops when an op isn’t implemented.\n\nEnjoy!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frikulauttia%2Fai-gpu-playground-mac","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frikulauttia%2Fai-gpu-playground-mac","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frikulauttia%2Fai-gpu-playground-mac/lists"}