https://github.com/webdevtodayjason/vrambudget
vrambudget.com — what LLM fits my hardware? VRAM math + 40 GPUs + 20 models + AVL L3
https://github.com/webdevtodayjason/vrambudget
Last synced: 17 days ago
JSON representation
vrambudget.com — what LLM fits my hardware? VRAM math + 40 GPUs + 20 models + AVL L3
- Host: GitHub
- URL: https://github.com/webdevtodayjason/vrambudget
- Owner: webdevtodayjason
- License: mit
- Created: 2026-05-21T02:25:44.000Z (about 2 months ago)
- Default Branch: main
- Last Pushed: 2026-05-21T21:35:03.000Z (about 2 months ago)
- Last Synced: 2026-06-05T07:18:49.616Z (about 1 month ago)
- Language: TypeScript
- Size: 854 KB
- Stars: 1
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
**What LLM fits my hardware?**
The math behind local LLM memory budgets. Plug in a GPU and a context length; see what's actually left for weights after the KV cache, runtime overhead, and a safety margin.
[Live site](https://vrambudget.com) · [Run the calculator](https://vrambudget.com/calc) · [Read the math](https://vrambudget.com/the-math) · [Join the discussion](https://github.com/webdevtodayjason/vrambudget/discussions)
40
GPU presets
20
curated models
9
quant formats
64
agent-readable routes
MIT
open source
## The thesis
Every "can I run this LLM" tool lists models and calls it done. They show a green checkmark next to a 70B on a 24GB card and hope you don't ask why. vrambudget skips the magic. There is one formula and three taxes:
```
VRAM ~= params × (bits ÷ 8)
```
That's the floor. Everything else is overhead.
```
weights_budget = total_vram × (1 − safety)
− kv_cache × concurrency
− framework_overhead
```
Three subtractions and one multiplication. Anyone telling you it needs to be more complicated is selling you something.
## How to use the calculator
The calculator is on the home page. Five sliders, four tiles, one bar, three tabs.
### 1. Pick a GPU
Across the top of the calculator: eight family tabs (RTX 50, RTX 40, RTX 30, Apple Silicon, Workstation, Datacenter, AMD, Intel). Each tab reveals every card in that family. Click a card to lock the calculator to its stock VRAM.
For a long-form view of every GPU at once, visit [/gpu](https://vrambudget.com/gpu).
### 2. Tune the four sliders
| Slider | What it does |
|---|---|
| **VRAM** | Total memory on the card. Auto-set from your GPU choice; override freely. |
| **System RAM** | Reserved for future model-split workflows. Not currently in the math. |
| **Context** | Token window. 8K is conservative; 32K and 128K make KV cache the dominant tax. |
| **Concurrency** | How many requests serve in parallel. Each one gets its own KV cache. |
| **Safety headroom** | Percentage of VRAM you refuse to spend. 15 percent is a sane default. |
### 3. Read the four tiles
- **Total VRAM** — device capacity
- **KV cache** — what the context-tax costs you
- **Runtime overhead** — CUDA / kernels / allocator slack
- **Weights budget** — what's actually left for the model
The orange tile is the answer. The budget bar below shows the four pieces stacked so you can see where the VRAM went.
### 4. Find a model
Three tabs below the bar:
- **Curated picks** — the 30 catalog models, grouped by fit (fits / tight / over). Each row picks the highest-bit quantization that fits your budget. Click any model name to open its Hugging Face repo.
- **Search Hugging Face** — live search across HF's GGUF-tagged models. Type a name; results render with the same fit classification keyed to your current budget.
- **By size** — drag a params slider; see all nine quants in a 3×3 matrix with the size and fit class for each.
### 5. Drill into a card or a model
Every GPU card in the grid has a `↗` link to its detail page. Each detail page pre-binds the calculator to that card, plus a fit table of the top 12 models, plus four "compare to" cards for stepping up or down by VRAM.
Same for models: every model name links to `/model/` with the GPU-recommendation matrix across five quants.
## What's in the catalogs
### GPUs (40)
`RTX 50`: 5070 · 5070 Ti · 5080 · 5090
`RTX 40`: 4060 · 4060 Ti 16GB · 4070 · 4070 Super · 4070 Ti Super · 4080 · 4080 Super · 4090
`RTX 30`: 3060 · 3070 · 3080 · 3080 Ti · 3090 · 3090 Ti
`Apple Silicon`: M2 Max 64 · M2 Ultra 192 · M3 Max 64 · M3 Max 96 · M3 Ultra 512 · M4 Pro 64 · M4 Max 128 · M5 Pro 64 · M5 Max 128
`Workstation`: A6000 · RTX 6000 Ada · RTX Pro 6000 · L40S
`Datacenter`: H100 80GB · H200 · B200 · DGX Spark · 2× H100 NVL
`AMD`: RX 7900 XTX · W7900 · MI300X
`Intel`: Arc B580 · Arc Pro B60 · Gaudi 3
### Models (30 curated, refreshed 2026-05)
`Meta`: Llama 3.2 1B · Llama 3.2 3B · Llama 3.1 8B · Llama 3.3 70B · Llama 3.1 405B
`Alibaba`: Qwen 2.5 7B · Qwen 2.5 32B · Qwen 2.5 Coder 32B · Qwen 2.5 72B · Qwen3 30B A3B · Qwen 3.5 9B · Qwen 3.6 27B · Qwen 3.6 35B A3B
`OpenAI`: gpt-oss 20B · gpt-oss 120B
`Mistral`: Mistral 7B v0.3 · Mistral Small 3 · Mixtral 8x7B · Mixtral 8x22B
`Microsoft`: Phi-4 Mini · Phi-4
`Google`: Gemma 2 9B · Gemma 4 E4B · Gemma 4 26B A4B
`DeepSeek`: DeepSeek V3 · DeepSeek R1
`Cohere`: Command R+
`IBM`: Granite 8B Code
`BigCode`: StarCoder2 15B
`01.AI`: Yi 34B
### Quantization formats (9)
`FP16/BF16` · `FP8/INT8` · `Q8_0` · `Q6_K` · `Q5_K_M` · `Q4_K_M` · `Q3_K_M` · `AWQ` · `GPTQ`
### LLM hosting runtimes (5)
`Cross-platform`: Ollama · LM Studio
`Apple Silicon`: MLX · oMLX
`Server`: vLLM
## Keeping the catalog fresh
LLMs ship faster than any curated list can keep up. vrambudget runs three feedback loops at once:
1. **Editorial curation** of `src/lib/models.ts` (~quarterly). The pinned list is canonical orgs only: `meta-llama`, `Qwen`, `openai`, `mistralai`, `microsoft`, `google`, `deepseek-ai`, `ibm-granite`, `bigcode`, `01-ai`, `CohereForAI`. Each entry carries a `releaseQuarter` so it's obvious what's stale. Forks, distills, and uncensored remixes belong on Hugging Face, not here.
2. **Live "Search Hugging Face" tab** in the calculator. Calls `https://huggingface.co/api/models?search=&filter=gguf&sort=downloads` directly from the browser. Always current; classifies each result against your VRAM budget on the fly.
3. **Build-time HF enrichment** (Phase 2, not shipped yet). A `scripts/refresh-hf-stats.ts` will fetch `downloads`, `likes`, and `last_modified` for every curated `hfRepo` at build, write the result into `data/hf-stats.json`, and surface it on each model detail page. Entries below a download threshold will get a build warning so they get reviewed in the next editorial pass.
**How to propose a new model:**
1. Confirm it's canonical (an official org repo, not a community quant)
2. Open a PR to `src/lib/models.ts` with `slug`, `name`, `hfRepo`, `params`, `family`, `type`, `contextK`, `fp16GB`, `summary`, `releaseQuarter`
3. If MoE, set `activeParams` to active-per-forward-pass count
4. CI runs `pnpm build` to validate the static export still emits cleanly
## Agent View Layer (AVL)
Every page on vrambudget ships a parallel agent-readable view at `.agent` (e.g. [`/gpu/rtx-4090.agent`](https://vrambudget.com/gpu/rtx-4090.agent)). The view is TOON-encoded and includes `@meta`, `@intent`, `@state`, `@actions`, `@context`, and `@nav` sections per the [AVL L3 conformance spec](https://agentviewlayer.org). The full manifest lives at [/agent.txt](https://vrambudget.com/agent.txt).
Built with [@frontier-infra/avl](https://github.com/frontier-infra/avl).
## Local development
```bash
pnpm install
pnpm dev # localhost:3000, hot reload
pnpm typecheck # tsc --noEmit
pnpm build # static export to out/, plus postbuild AVL emitter
pnpm start # serve out/ on $PORT (default 3000)
```
`out/` after build contains:
- 64 route HTMLs (home, the-math, /gpu, /model, 40 GPU detail, 20 model detail)
- 64 `.agent` companion files
- `agent.txt`, `sitemap.xml`, `robots.txt`
- 61 OG image PNGs, favicon SVG, apple-touch-icon
- Static assets and font bundles
## Deploy
The project is wired for **Railway** out of the box. `railway.toml` pins Node 22 + pnpm 9 and configures Nixpacks with explicit build/start commands and a `/` healthcheck.
Works on any static host: Cloudflare Pages, Vercel, Netlify, S3+CloudFront, GitHub Pages, plain nginx. Just point at `out/`.
## Stack
- [Next.js 15](https://nextjs.org) (App Router, static export)
- [TypeScript 5](https://www.typescriptlang.org)
- [React 18](https://react.dev)
- [pnpm 9](https://pnpm.io)
- Vanilla CSS · [Geist](https://vercel.com/font) + [JetBrains Mono](https://www.jetbrains.com/lp/mono/)
- [@frontier-infra/avl](https://github.com/frontier-infra/avl) for the parallel agent views
- [Playwright](https://playwright.dev) for end-to-end browser verification
## Roadmap
- **Phase 2: Runtime badges.** Per-model compatibility badges for Ollama, LM Studio, vLLM, and MLX with one-click install commands.
- **Build-time HF enrichment.** Pull download counts, license, and architecture from the HF Hub API at build so curated cards show context beyond what the slug carries.
- **Per-model architecture.** Replace the 13B reference for KV cache with per-model GQA / MQA + head dimensions for tighter estimates.
- **Tokens-per-second.** Bandwidth-based throughput estimates next to the fit pills.
- **Cloud comparison.** This model on your machine vs OpenRouter pricing for the equivalent.
See [issues](https://github.com/webdevtodayjason/vrambudget/issues) for the live list.
## Contributing
Open a PR. Two soft rules:
- **No em dashes** in new copy. Colons, periods, semicolons.
- **Numbers in monospace.** VRAM, params, GB, percentages.
## About the builder
Built by **Jason Brashear**. MSP owner, AI application developer, and 30-year IT veteran out of Texas.
- Personal site: [jasonbrashear.com](https://jasonbrashear.com/)
- Substack: [jasonbrashear.substack.com](https://jasonbrashear.substack.com/)
- Medium: [@jason_81067](https://medium.com/@jason_81067)
- Dev.to: [@webdevtodayjason](https://dev.to/webdevtodayjason)
- X: [@argentAIOS](https://x.com/argentAIOS)
- GitHub: [@webdevtodayjason](https://github.com/webdevtodayjason)
## License
MIT. See [LICENSE](LICENSE). Use it however you want.