Projects in Awesome Lists tagged with distributed-inference
A curated list of projects in awesome lists tagged with distributed-inference .
https://github.com/flashinfer-ai/flashinfer
FlashInfer: Kernel Library for LLM Serving
attention cuda distributed-inference gpu jit large-large-models llm-inference moe nvidia pytorch
Last synced: 13 Jun 2026
https://github.com/gpustack/gpustack
A GPU cluster manager that configures and orchestrates inference engines like vLLM and SGLang for high-performance AI model deployment.
ascend cuda deepseek distributed-inference genai high-performance-inference inference llama llm llm-inference llm-serving maas mindie openai qwen rocm sglang vllm
Last synced: 20 Apr 2026
https://github.com/adt109119/llamacpp-distributed-inference
一個基於 llama.cpp 的分佈式 LLM 推理程式，讓您能夠利用區域網路內的多台電腦協同進行大型語言模型的分佈式推理，使用 Electron 的製作跨平台桌面應用程式操作 UI。
distributed-inference distributed-llm gguf llamacpp llm llm-inference rpc
Last synced: 23 May 2026
https://github.com/clearclown/tirami
Tirami — distributed LLM inference where compute is currency. 1 TRM = 10^9 FLOPs. 21B supply cap, yield halving, staking, collusion resistance. 100% Rust, OpenAI-compatible, no token, no ICO. "tira mi su" = pull me up.
bitcoin compute-economy distributed-inference llama-cpp llm nostr openai-api p2p rust self-improving-agents tirami tokenomics trm
Last synced: 15 Jun 2026
https://github.com/zerogpu/cli
Terminal CLI for the ZeroGPU Orchestration API — classify, summarize, redact PII, extract entities, and chat from your shell
ai-cli ai-inference distributed-inference edge-inference gliner iab-classification inference-cli llm-cli named-entity-recognition nano-language-models openai-compatible pii-detection pii-redaction slm small-language-models structured-extraction text-summarization typescript zero-shot-classification zerogpu
Last synced: 12 Jun 2026
