https://github.com/nagababumo/efficiently-serving-llms

batching continuous-batching lora lorax low-rank-adaptation multi-lora quantization

Last synced: 29 Oct 2025

https://github.com/ifuryst/nanollmserve

🌱 A tiny, readable LLM serving engine with vLLM/SGLang-style features.

ai-infra continuous-batching fastapi inference kv-cache llm llm-engine llm-inference llm-infra llm-serving lora openai-compatible paged-attention pytorch quantization sglang speculative-decoding teaching transformers vllm

Last synced: 05 Jun 2026