Projects in Awesome Lists tagged with gpu-inference
A curated list of projects in awesome lists tagged with gpu-inference .
https://github.com/brontoguana/krasis
Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware
cpu-inference gguf-model-support gpu-inference high-performance-inference hybrid-inference inference-engine inference-optimization large-language-models llama-cpp-alternative llm-inference mixture-of-experts transformer
Last synced: 30 May 2026
https://github.com/mpaepper/docker-cifar
Docker based GPU inference of machine learning models
deep-learning gpu-inference pytorch
Last synced: 09 Apr 2025
https://github.com/mlorentedev/pollex
Text polishing API (Go) + Chrome extension + llama.cpp GPU inference on Jetson Nano. Self-hosted, private, fast.
chrome-extension go gpu-inference jetson-nano llama-cpp llm self-hosted text-processing
Last synced: 05 Mar 2026
https://github.com/manishklach/gb300-rl-runtime
Close-to-metal C/CUDA lab for RL inference fast paths: persistent GPU workers, hugepage KV arenas, cacheline-aware command rings, and async reward handoff. Goal: remove page faults, malloc/free, scheduler wakeups, CPU round-trips, and KV migration from the per-token path.
ai-infrastructure close-to-metal cuda gb300 gpu-inference hpc lock-free nvlink reinforcement-learning spsc-queue
Last synced: 09 Jun 2026
