GPU / VRAM calculator for local LLMs
How much VRAM does your model need? Pick parameter count, quantization, and context length — get weight + KV-cache estimates and a fit verdict for common GPUs and Apple Silicon tiers.
Model size
Quantization
Context length
KV cache grows with context. Long chats and RAG can dominate VRAM on smaller cards.
Estimated VRAM needed
KV factor: per 1k context at this model size. Simplified — real usage depends on architecture (GQA, MLA) and runtime.
Show your work
| Component | Formula | GB |
|---|---|---|
| Weights (raw) | ||
| Loading overhead | ||
| KV cache | ||
| Total |
Hardware fit
snapshot. Apple tiers assume ~72% of unified memory is usable for GPU.
| Device | Usable | Verdict | Headroom |
|---|---|---|---|
Q4_K_M is the usual quality/size sweet spot for local models. Offloading layers to CPU works but is slow. On Apple Silicon, unified memory bandwidth often matters more than the headline GB number. Partial GPU offload and smaller context are the first knobs when you are tight on VRAM.
About this tool
Running Llama, Mistral, or Qwen locally means fitting quantized weights plus a growing KV cache into VRAM (or unified memory). This calculator uses transparent approximations — not vendor-specific profilers — so you can sanity-check before you download a 70B GGUF.
Pair it with the local vs API break-even calculatorif you are deciding whether to buy hardware at all.