https://github.com/Luce-Org/lucebox-hub

Lucebox: LLM inference server built for speed for specific consumer hardware.

cuda cuda-kernels dflash kernel llama-cpp local-ai luce lucebox megakernel nvidia-cuda pflash qwen rtx3090 speculative-decoding speculative-prefill

Last synced: 23 May 2026

https://github.com/hec-ovi/vllm-awq4-qwen

vLLM Qwen 3.6-27B (AWQ-INT4) + DFlash speculative decoding on AMD Strix Halo (gfx1151 iGPU, 128 GB UMA, ROCm 7.13). 24.8 t/s single-stream, vision, tool calling, 256K context, OpenAI-compatible, Docker. Matches DGX Spark FP8+DFlash+MTP at a third of the cost. No CUDA.

27b amd-strix-halo awq dflash docker gfx1151 llm-inference multimodal-llm openai-api qwen3 rdna35 rocm ryzen-ai-max speculative-decoding vllm

Last synced: 21 Jun 2026

https://github.com/croll83/llama.cpp-dgx

llama.cpp fork optimized for NVIDIA DGX Spark / GB10 (Blackwell, SM 12.1) — TurboQuant weights + KV, NVFP4, DFlash MTP

blackwell dflash gb10 llama-cpp nvfp4 speculative-decoding turboquant

Last synced: 02 Jun 2026

https://github.com/jcartu/repne-dflash-newimage

benchmark blackwell dflash qwen vllm

Last synced: 12 Jun 2026

https://github.com/jcartu/qwen-bench-2026-05-11-v2-followup

Study #4: FP8+MTP{3,5} speed on repne/vllm:v2 + max_tokens=8192 quality re-runs for BF16+DFlash n=8 and FP8+MTP=3. Follow-up to studies #2 and #3.

benchmark blackwell dflash humaneval inference mbpp mtp qwen-bench qwen3 speculative-decoding vllm

Last synced: 12 Jun 2026