https://github.com/nerdalert/vllm-benchmark-graphs
Graphing vLLM Benchmarks
https://github.com/nerdalert/vllm-benchmark-graphs
Last synced: 11 months ago
JSON representation
Graphing vLLM Benchmarks
- Host: GitHub
- URL: https://github.com/nerdalert/vllm-benchmark-graphs
- Owner: nerdalert
- Created: 2025-04-10T21:30:05.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2025-04-17T03:01:17.000Z (about 1 year ago)
- Last Synced: 2025-06-11T22:43:24.979Z (about 1 year ago)
- Language: Python
- Homepage:
- Size: 20.5 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# VLLM Benchmark Graphing
### Run
With `results.json` in the same directory (or use the included sample) run:
```commandline
python3.12 -m venv venv
source venv/bin/activate
pip install --upgrade pip
pip install pandas plotly kaleido
python prompt-comparisons.py --export html
python metric-perfs.py --export html
```
For static png graphs run:
```commandline
python prompt-comparisons.py --export png
python metric-perfs.py --export png
```
# E2E Automation
For CI run `e2e-bench.sh` to builds all dependencies and run both tests. Flags also have corresponding ENVs.
```commandline
./e2e-bench.sh --port 8000 --model meta-llama/Llama-3.2-1B --cuda-device 0
```