Projects in Awesome Lists tagged with cpu-inference
A curated list of projects in awesome lists tagged with cpu-inference .
https://github.com/kennethleungty/Llama-2-Open-Source-LLM-CPU-Inference
Running Llama 2 and other Open-Source LLMs on CPU Inference Locally for Document Q&A
c-transformers chatgpt cpu cpu-inference deep-learning document-qa faiss langchain language-models large-language-models llama llama-2 llm machine-learning natural-language-processing nlp open-source-llm python sentence-transformers transformers
Last synced: 09 May 2025
https://github.com/kennethleungty/llama-2-open-source-llm-cpu-inference
Running Llama 2 and other Open-Source LLMs on CPU Inference Locally for Document Q&A
c-transformers chatgpt cpu cpu-inference deep-learning document-qa faiss langchain language-models large-language-models llama llama-2 llm machine-learning natural-language-processing nlp open-source-llm python sentence-transformers transformers
Last synced: 16 May 2025
https://github.com/brontoguana/krasis
Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware
cpu-inference gguf-model-support gpu-inference high-performance-inference hybrid-inference inference-engine inference-optimization large-language-models llama-cpp-alternative llm-inference mixture-of-experts transformer
Last synced: 30 May 2026
https://github.com/rbitr/llm.f90
LLM inference in Fortran
ai chatbot cpu-inference language-model llama llama2 llamacpp llm mamba phi-2 state-space-model transformer
Last synced: 05 Oct 2025
https://github.com/gabriele-mastrapasqua/qwen3-tts
Pure C inference engine for Qwen3-TTS text-to-speech. No Python, no PyTorch — just C and BLAS. Supports 0.6B and 1.7B models, 9 voices, 10 languages.
c cpu-inference inference inference-engine multilingual pure-c qwen qwen3-tts simd speech-synthesis text-to-speech tts voiceclone voicecloning
Last synced: 01 Apr 2026
https://github.com/jozsefszalma/homelab
The bare metal in my basement
ai bare-metal cpu-inference deep-learning gpu hardware-hacking hobby-project homelab machine-learning server
Last synced: 28 Dec 2025
https://github.com/yybit/pllm
Portable LLM - A rust library for LLM inference
Last synced: 10 Apr 2025
https://github.com/laelhalawani/gguf_llama
Wrapper for simplified use of Llama2 GGUF quantized models.
cpu-inference gguf llama llama2 llamacpp quantization
Last synced: 10 Mar 2026
https://github.com/codito/arey
Simple large language model playground app
ai cli cpu-inference gguf large-language-models llama2 llamacpp llm local-model mistral
Last synced: 10 Mar 2026
https://github.com/johnclaw/chatllm.v
V-lang api wrapper for llm-inference chatllm.cpp
api-wrapper bindings chatbot chatllm cpu-inference gemma ggml inference llama llm llm-inference llms mistral phi3 quantization qwen v-lang vlang
Last synced: 20 Jun 2025
https://github.com/johnclaw/chatllm.vb
VB.NET api wrapper for llm-inference chatllm.cpp
api-wrapper bindings chatllm cpu-inference gemma ggml int8 int8-inference int8-quantization llama llm-inference mistral qwen vb-net vbnet
Last synced: 05 Jul 2025
https://github.com/johnclaw/chatllm.cs
C# api wrapper for llm-inference chatllm.cpp
api-wrapper bindings chatllm cpu-inference csharp gemma ggml inference int8 int8-inference int8-quantization llama llm llm-inference llms mistral qwen
Last synced: 11 Feb 2026
https://github.com/xxxn3m3s1sxxx/atlas-tq1_0
TQ1.0 ternary inference engine for BitNet b1.58 on CPU. Pack + run Falcon3-1B/3B/7B/10B, no GPU needed.
1bit-llm avx2 bitnet bitnet-b158 bonsai cpp cpu-inference edge-ai falcon3 high-performance inference-engine llm-inference local-llm quantization simd ternary ternary-weights
Last synced: 31 May 2026
https://github.com/johnclaw/chatllm.nim
Nim api-wrapper for llm-inference chatllm.cpp
api-wrapper bindings chatbot chatllm cpu-inference gemma ggml inference llama llm llm-inference llms mistral nim nim-lang nim-language nimlang phi quantization qwen
Last synced: 09 Feb 2026
https://github.com/chinese-soup/cbot-telegram-whisper
Simple bot that transcribes Telegram voice messages. Powered by go-telegram-bot-api & whisper.cpp Go bindings.
bot cpu-inference golang openai speech-recognition speech-to-text whisper whisper-cpp whispercpp
Last synced: 11 May 2026
https://github.com/johnclaw/chatllm.d
D-lang api wrapper for llm-inference chatllm.cpp
api-wrapper bindings chatbot chatllm cpu-inference d-lang d-language dlang gemma ggml inference llama llm llm-inference llms mistral quantization qwen
Last synced: 29 Apr 2026
https://github.com/johnclaw/chatllm.lua
lua api wrapper for llm-inference chatllm.cpp
api-wrapper bindings chatbot chatllm cpu-inference gemma ggml inference llama llm llm-inference llms lua luajit mistral quantization qwen
Last synced: 11 May 2026
https://github.com/johnclaw/llama-3.2-1b.vb
one-file llama 3.2 1b fp16 cpu inference in pure vb.net
basic-programming cpu-inference fp16 inference inference-engine llama llama3 llama3-2 llm llm-inference llm-serving llms vb-net vbnet visual-basic-dot-net visual-basic-net
Last synced: 01 May 2026
https://github.com/johnclaw/chatllm.rs
rust api wrapper for llm-inference chatllm.cpp
api-wrapper bindings chatbot chatllm cpu-inference gemma ggml inference llama llm llm-inference llms mistral quantization qwen rust
Last synced: 29 Apr 2026
https://github.com/johnclaw/gemma-2-2b-it.cs
gemma-2-2b-it int8 cpu inference in one file of pure C#
cpu-inference csharp gemma gemma2 gemma2-2b-it inference inference-engine int8 int8-inference int8-quantization llm llm-inference llm-serving llms quantization
Last synced: 19 May 2026
https://github.com/johnclaw/chatllm.kt
kotlin api wrapper for llm-inference chatllm.cpp
api-wrapper bindings chatbot chatllm cpu-inference gemma ggml inference kotlin llama llm llm-inference llms mistral quantization qwen
Last synced: 12 Apr 2026
https://github.com/facex-engine/facex
Face verification in the browser. 74 KB WebAssembly. No server, no cloud, no dependencies. Also runs native at 3ms on CPU.
avx-512 avx2 biometrics c99 computer-vision cpu-inference deep-learning edge-ai embedded-ai face-embedding face-recognition face-verification low-latency onnx-runtime production-ready simd zero-dependencies
Last synced: 01 May 2026
https://github.com/asherk7/memo
Lightweight multimodal emotion recognition
cpu-inference edge-ai facial-expression-recognition knowledge-distillation late-fusion lora mediapipe minilm ml mobilenetv3 multimodal peft pytorch sentiment-analysis speech-emotion-recognition transformers
Last synced: 20 Jun 2026
https://github.com/nmicic/int-llm
Integer-only LLM stack in pure C — run TinyLlama and train a tiny GPT in Q16.48 fixed-point, zero floating point. A reproducible fun experiment.
cpu-inference deterministic fixed-point from-scratch inference-engine integer-arithmetic llm machine-learning pure-c quantization reproducibility tinyllama
Last synced: 02 Jul 2026
https://github.com/dragoshont/apprenticeops
Open, reproducible benchmark for small, quantized, fully-offline local LLMs as homelab ops assistants — profiling quality × safety × energy together and reducing model choice to a measured Pareto front. arXiv preprint → NeurIPS D&B track. Human-guided, AI-assisted.
aiops cpu-inference edge-ai energy-efficiency homelab llm-benchmark llm-safety local-llm ollama quantization reproducibility small-language-models
Last synced: 14 Jul 2026
https://github.com/usr-wwelsh/turbolab
Minimal local AI gateway with web UI, HuggingFace search, TurboQuant, and OpenAI API. Single binary, no cloud, no API keys.
ai-gateway cpu-inference gguf go huggingface llama-cpp local-ai openai-api safetensors self-hosted svelte
Last synced: 30 May 2026
https://github.com/hankthebldr/local-ai-platform
Self-hosted LLM infrastructure with OpenAI-compatible API. CPU-optimized. Source-available.
ai-inference cpu-inference enclave llm local-ai macos ollama openai-compatible privacy self-hosted
Last synced: 31 May 2026
https://github.com/bhimanbaghel/llama-streamlit-app
🤖 AI Text Completion App built with Streamlit and Llama-3.2-1B. Generate creative text completions with an intuitive web interface. GPU & CPU optimized, easy to deploy, perfect for content creation and AI experimentation.
ai cpu-inference huggingface llama machine-learning nlp python streamlit streamlit-webapp text-generation transformers webapp
Last synced: 11 May 2026
