{"id":50330376,"url":"https://github.com/themactep/ai-benchmarks","last_synced_at":"2026-05-29T09:02:46.180Z","repository":{"id":354943965,"uuid":"1226144077","full_name":"themactep/ai-benchmarks","owner":"themactep","description":"A set of scripts to prep your hardware for AI","archived":false,"fork":false,"pushed_at":"2026-05-01T03:14:39.000Z","size":22,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2026-05-23T19:31:13.506Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/themactep.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-01T02:54:18.000Z","updated_at":"2026-05-01T03:14:43.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/themactep/ai-benchmarks","commit_stats":null,"previous_names":["themactep/ai-benchmarks"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/themactep/ai-benchmarks","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/themactep%2Fai-benchmarks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/themactep%2Fai-benchmarks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/themactep%2Fai-benchmarks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/themactep%2Fai-benchmarks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/themactep","download_url":"https://codeload.github.com/themactep/ai-benchmarks/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/themactep%2Fai-benchmarks/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33644313,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-05-29T02:00:06.066Z","response_time":107,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-05-29T09:02:45.151Z","updated_at":"2026-05-29T09:02:46.166Z","avatar_url":"https://github.com/themactep.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AI Benchmarks\n\nBenchmark suite for the Intel Core Ultra 9 185H (Meteor Lake) AI PC.\n\n**Accelerators**: CPU (AVX2 + VNNI), Intel Arc iGPU, Intel AI Boost NPU (11 TOPS INT8), AMD Radeon RX 580 (Vulkan)\n\n## Quick start\n\n```bash\n# Run everything\n./run-all.sh\n\n# Or individually:\npython3 01-system-info.py       # Hardware detection\npython3 02-pytorch-cpu.py       # PyTorch CPU benchmarks\npython3 03-openvino-all.py      # OpenVINO CPU / GPU / NPU\nbash 04-llama-cpp-check.sh      # llama.cpp Vulkan devices\n```\n\nRun a specific device only:\n```bash\npython3 03-openvino-all.py --device NPU\npython3 03-openvino-all.py --device GPU\npython3 03-openvino-all.py --matmul-size 2048\n```\n\n## Expected results (this system)\n\n| Device | Matmul 4096² f32 | CNN FPS | Best for |\n|--------|-----------------|---------|----------|\n| CPU    | ~0.2 TFLOPS     | ~32     | Training, float32 inference |\n| GPU    | ~0.7 TFLOPS     | ~65     | Vision models, OpenVINO |\n| NPU    | ~1.2 TFLOPS     | ~58     | Low-power INT8 inference |\n\n**PyTorch CPU** (with IPEX + perf governor): ~0.57 TFLOPS matmul, ~493 GFLOPS peak at 16 threads.\n\n**llama.cpp Vulkan LLM inference**:\n- Intel Arc: 48 GB shared → run 70B models\n- AMD RX 580: 8 GB VRAM → 7B-13B models at ~20-40 tok/s\n\n## Prerequisites\n\nAll packages are already installed. If setting up from scratch:\n\n```bash\n# Core AI libraries\npip install torch intel-extension-for-pytorch openvino numpy\n\n# Intel GPU compute (OpenVINO GPU plugin needs this)\nsudo apt install intel-opencl-icd libze-intel-gpu1\n\n# Intel NPU (downloaded from GitHub)\n# 1. Level Zero NPU driver:\n#    https://github.com/intel/linux-npu-driver/releases\n#    → intel-level-zero-npu_*.deb\n# 2. NPU compiler:\n#    https://github.com/openvinotoolkit/npu_compiler/releases\n#    → extract lib → copy to openvino/libs/libopenvino_intel_npu_compiler.so\n\n# CPU governor (already set to performance)\necho performance | sudo tee /sys/devices/system/cpu/cpufreq/policy*/scaling_governor\n\n# User must be in render group for NPU access\nsudo usermod -a -G render $USER\n# Log out and back in for this to take effect\n\n# llama.cpp with Vulkan\ncd ~/llama.cpp\ncmake -B build -DGGML_VULKAN=ON\ncmake --build build --config Release -j$(nproc)\n```\n\n## Architecture notes\n\n- **CPU**: 6 P-cores (4.8-5.1 GHz) + 8 E-cores (2.5-3.8 GHz) + 2 LP E-cores. No AMX (datacenter-only). Has AVX-VNNI for INT8 acceleration. **bfloat16 is extremely slow** — use float32 or INT8 instead. Pin PyTorch to P-cores with `torch.set_num_threads(8)` or `taskset -c 0-11`.\n\n- **GPU (Arc)**: Integrated GPU sharing system RAM (48 GB max). Access via OpenVINO GPU plugin or llama.cpp Vulkan backend.\n\n- **NPU (AI Boost)**: Dedicated AI accelerator, ~11 TOPS INT8. Best for sustained low-power INT8 inference. Requires Level Zero NPU driver + OpenVINO NPU compiler.\n\n- **GPU (RX 580)**: Discrete Polaris GPU, 8 GB VRAM. No ROCm support. Use via Vulkan (llama.cpp only).\n\n## File structure\n\n```\nai-benchmarks/\n  01-system-info.py       Hardware detection, drivers, instruction sets\n  02-pytorch-cpu.py       Matmul, thread scaling, transformer sim, mem bw\n  03-openvino-all.py      CPU vs GPU vs NPU (matmul + CNN)\n  04-llama-cpp-check.sh   Vulkan device listing for LLM inference\n  run-all.sh              Run all benchmarks sequentially\n  README.md               This file\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthemactep%2Fai-benchmarks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthemactep%2Fai-benchmarks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthemactep%2Fai-benchmarks/lists"}