https://github.com/slinusc/bench360

Bench360 is a modular benchmarking suite for local LLM deployments. It offers a full-stack, extensible pipeline to evaluate the latency, throughput, quality, and cost of LLM inference on consumer and enterprise GPUs. Bench360 supports flexible backends, tasks and scenarios, enabling fair and reproducible comparisons for researchers & practitioners.

bench360 benchmark deployment energy energy-consumption engine framework inference llm llm-inference local mldeploy optimization performance quantization sglang tgi vllm

Last synced: 28 Apr 2026