Projects in Awesome Lists tagged with distributed-inference
A curated list of projects in awesome lists tagged with distributed-inference .
https://github.com/flashinfer-ai/flashinfer
FlashInfer: Kernel Library for LLM Serving
attention cuda distributed-inference gpu jit large-large-models llm-inference moe nvidia pytorch
Last synced: 08 May 2026
https://github.com/gpustack/gpustack
A GPU cluster manager that configures and orchestrates inference engines like vLLM and SGLang for high-performance AI model deployment.
ascend cuda deepseek distributed-inference genai high-performance-inference inference llama llm llm-inference llm-serving maas mindie openai qwen rocm sglang vllm
Last synced: 20 Apr 2026
https://github.com/adt109119/llamacpp-distributed-inference
一個基於 llama.cpp 的分佈式 LLM 推理程式，讓您能夠利用區域網路內的多台電腦協同進行大型語言模型的分佈式推理，使用 Electron 的製作跨平台桌面應用程式操作 UI。
distributed-inference distributed-llm gguf llamacpp llm llm-inference rpc
Last synced: 16 Apr 2026
