How much VRAM do I need for a 70B model?

A 70B model at Q4_K_M needs roughly 40-44 GB; at Q8_0 about 75 GB. Two 24 GB cards can run it at Q4 with a moderate context.

Does quantization lower VRAM use?

Directly: 4-bit weights are ~1/3 the size of FP16. The table's quantization switch shows the same model at Q4, Q8 and FP16.

Is the estimate exact?

No — it covers weights, a moderate-context KV cache and runtime overhead. Long contexts or large batches raise the real number.

VRAM Calculator

What can your GPU actually run? Pick a card's memory and a quantization — the table shows which local models fit, and by how much.

70 fit comfortably · 7 tight on 12 GB at Q4_K_M (4-bit)

Model	Params	Est. VRAM	Verdict
MobileLLM-125M	125M	0.9 GB	Fits
SmolLM2-135M-Instruct	135M	0.9 GB	Fits
LFM2.5-230M	230M	0.9 GB	Fits
LFM2-350M	350M	1.0 GB	Fits
MobileLLM-350M	350M	1.0 GB	Fits
LFM2.5-350M	350M	1.0 GB	Fits
SmolLM2-360M-Instruct	360M	1.0 GB	Fits
Qwen2.5-0.5B-Instruct	500M	1.1 GB	Fits
Qwen3.5-0.8B-MTP	800M	1.3 GB	Fits
Qwen3.5-0.8B	800M	1.3 GB	Fits
gemma-3-1b-it	1B	1.4 GB	Fits
Llama-3.2-1B-Instruct	1B	1.4 GB	Fits
LFM2.5-1.2B-Instruct	1B	1.4 GB	Fits
Qwen3-0.6B	1B	1.4 GB	Fits
MiniCPM5-1B	1B	1.4 GB	Fits
Spark-X2.5-1.7B	2B	2.1 GB	Fits
Qwen3-1.7B	2B	2.1 GB	Fits
LFM2.5-2.6B	3B	2.7 GB	Fits
Llama-3.2-3B-Instruct	3B	2.7 GB	Fits
VibeThinker-3B	3B	2.7 GB	Fits
Ministral-3-3B-Instruct-2512	3B	2.7 GB	Fits
OpenLLaMA-3B	3B	2.7 GB	Fits
Qwen3.5-4B-MTP	4B	3.4 GB	Fits
Qwen3.8-4B-Distill	4B	3.4 GB	Fits
Spark-X2.5-4B	4B	3.4 GB	Fits
NVIDIA-Nemotron-3-Nano-4B	4B	3.4 GB	Fits
Qwen3.5-4B	4B	3.4 GB	Fits
Qwen3-4B-Thinking-2507	4B	3.4 GB	Fits
gemma-3n-E4B-it-text	4B	3.4 GB	Fits
Qwen3-4B	4B	3.4 GB	Fits
Phi-4-mini-reasoning	4B	3.4 GB	Fits
gemma-4-E2B-it	5B	4.0 GB	Fits
Falcon3-7B-Instruct	7B	5.3 GB	Fits
AgentTest-7B	7B	5.3 GB	Fits
Ling-3.0-tiny	8B	6.0 GB	Fits
Qwen3-8B	8B	6.0 GB	Fits
granite-4.2-8b	8B	6.0 GB	Fits
Qwen2.5-VL-7B-Instruct	8B	6.0 GB	Fits
DeepSeek-R1-0528-Qwen3-8B	8B	6.0 GB	Fits
Llama-3.1-8B-Instruct	8B	6.0 GB	Fits
LFM2.5-8B-A1B	8B	6.0 GB	Fits
OpenReasoning-Nemotron-7B	8B	6.0 GB	Fits
InternVL3-8B	8B	6.0 GB	Fits
gemma-4-E4B-it	8B	6.0 GB	Fits
Meta-Llama-3-8B-Instruct	8B	6.0 GB	Fits
DeepSeek-R1-Distill-Qwen-7B	8B	6.0 GB	Fits
AceReason-Nemotron-1.1-7B	8B	6.0 GB	Fits
LFM2-8B-A1B	8B	6.0 GB	Fits
WebWorld-14B	8B	6.0 GB	Fits
WebWorld-8B	8B	6.0 GB	Fits
Qwen3.5-9B-MTP	9B	6.6 GB	Fits
Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated-i1	9B	6.6 GB	Fits
Qwen3.8-9B-Distill	9B	6.6 GB	Fits
Qwythos-9B-v2	9B	6.6 GB	Fits
DeepSeek-V4-Pro-Qwen3.5-9B-MTP	9B	6.6 GB	Fits
Qwen3-VL-8B-Instruct	9B	6.6 GB	Fits
Ornith-1.0-9B	9B	6.6 GB	Fits
NVIDIA-Nemotron-Nano-9B-v2	9B	6.6 GB	Fits
Carnice-9b	9B	6.6 GB	Fits
Qwen3.5-9B	10B	7.3 GB	Fits
Ornith-1.5-9B	10B	7.3 GB	Fits
GLM-4.6V-Flash	10B	7.3 GB	Fits
gemma-4-12B-it-qat	12B	8.6 GB	Fits
gemma-4-12B-coder-fable5-composer2.5-v1	12B	8.6 GB	Fits
gemma-3-12b-it	12B	8.6 GB	Fits
gemma-4-12B-it	12B	8.6 GB	Fits
NVIDIA-Nemotron-Nano-12B-v2	12B	8.6 GB	Fits
Ministral-3-14B-Reasoning-2512	14B	9.9 GB	Fits
InternVL3-14B	14B	9.9 GB	Fits
Qwen3-14B	14B	9.9 GB	Fits
phi-4	15B	10.5 GB	Tight
Phi-4-reasoning-plus	15B	10.5 GB	Tight
Qwen2.5-Coder-14B-Instruct	15B	10.5 GB	Tight
Phi-4-reasoning	15B	10.5 GB	Tight
DeepSeek-R1-Distill-Qwen-14B	15B	10.5 GB	Tight
Nemotron-Cascade-14B-Thinking	15B	10.5 GB	Tight
DeepSeek-Coder-V2-Lite-Instruct	16B	11.2 GB	Tight
gpt-oss-20b	21B	14.4 GB	Too big
reka-flash-3	21B	14.4 GB	Too big
mistralai_Mistral-Small-3.2-24B-Instruct-2506	24B	16.3 GB	Too big
LFM2-24B-A2B	24B	16.3 GB	Too big
DeepHermes-3-Mistral-24B-Preview	24B	16.3 GB	Too big
Trinity-Mini	26B	17.6 GB	Too big
Qwen3.8-27B-MTP	27B	18.3 GB	Too big
gemma-4-26B-A4B-it-ultra-uncensored-heretic	27B	18.3 GB	Too big
gemma-4-26B-A4B-it	27B	18.3 GB	Too big
Gemma4-26B-A4B-QAT-Uncensored-HauhauCS-Balanced-MTP	27B	18.3 GB	Too big
gemma-4-26B-A4B-it-qat	27B	18.3 GB	Too big
Ornstein3.6-27B-MTP-NSC-ACE-SABER	27B	18.3 GB	Too big
Underdog-Saluki-27B	27B	18.3 GB	Too big
Ternary Bonsai 2 27B	27B	18.3 GB	Too big
Qwen3.8-27B	28B	18.9 GB	Too big
Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP	28B	18.9 GB	Too big
Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-YMQ	28B	18.9 GB	Too big
Qwen3.6-27B	28B	18.9 GB	Too big
Qwen3.8-27B-Uncensored	28B	18.9 GB	Too big
Qwen3.8-27B-Fable-Distill	28B	18.9 GB	Too big
Huihui-Qwen3.8-27B-abliterated	28B	18.9 GB	Too big
Bonsai-27B	28B	18.9 GB	Too big
Qwen3.8-27B-5090-goldilocks	28B	18.9 GB	Too big
Qwen3.8-27B-Ridge	28B	18.9 GB	Too big
Godoter-27B	28B	18.9 GB	Too big
Qwen3.6-27B-i1	28B	18.9 GB	Too big
Qwen3.6-27B-MTP	28B	18.9 GB	Too big
ThinkingCap-Qwen3.6-27B	28B	18.9 GB	Too big
Qwen3.5-27B	28B	18.9 GB	Too big
Tess-4-27B	28B	18.9 GB	Too big
granite-4.2-30b	29B	19.6 GB	Too big
Hy-MT2-30B-A3B	30B	20.2 GB	Too big
Nemotron-3-Nano-Omni-30B-A3B-Reasoning	30B	20.2 GB	Too big
Qwen3-Coder-30B-A3B-Instruct	30B	20.2 GB	Too big
Muse-Glimmer-30B	30B	20.2 GB	Too big
North-Mini-Code-1.0	30B	20.2 GB	Too big
Qwen3-30B-A3B-Thinking-2507	31B	20.9 GB	Too big
QUEST-30B-RL-B70-Turbo	31B	20.9 GB	Too big
Qwen3-30B-A3B-Instruct-2507	31B	20.9 GB	Too big
NVIDIA-Nemotron-3.5-Lightning-30B-A3B	32B	21.5 GB	Too big
NVIDIA-Nemotron-3-Nano-30B-A3B	32B	21.5 GB	Too big
Olmo-3-32B-Think	32B	21.5 GB	Too big
Nemotron-Cascade-2-30B-A3B	32B	21.5 GB	Too big
Laguna-XS-2.1	33B	22.1 GB	Too big
gemma-4-31B-it	33B	22.1 GB	Too big
GLM-Z1-32B-0414	33B	22.1 GB	Too big
Apodex-1.1-mini	35B	23.4 GB	Too big
Qwen3.6-35B-A3B-MTP	35B	23.4 GB	Too big
Qwen3.5-35B-A3B	35B	23.4 GB	Too big
KAT-Coder-V2.5-Dev-AWQ-W4A16-ASYM	35B	23.4 GB	Too big
Kwaipilot_KAT-Coder-V2.5-Dev	35B	23.4 GB	Too big
BTL-4-Compact	35B	23.4 GB	Too big
badtheorylabs_BTL-4	35B	23.4 GB	Too big
Ornith-1.0-35B-PrismaAURA-4.75bit-vllm-MTP	35B	23.4 GB	Too big
Ornith-1.0-35B-B70-Turbo	35B	23.4 GB	Too big
Nex-N2-mini-B70-Turbo	35B	23.4 GB	Too big
Ornith-1.0-35B	35B	23.4 GB	Too big
Qwen-AgentWorld-35B-A3B-B70-Turbo	35B	23.4 GB	Too big
MiniCPM-V 4.7-35B-A3B	35B	23.4 GB	Too big
Ornith-1.5-35B-A3B-MTP	36B	24.1 GB	Too big
Ornith-1.5-35B-A3B	36B	24.1 GB	Too big
Unsloth-Ornith-1.5-35B-A3B	36B	24.1 GB	Too big
Ornith-1.5-35B-A3B-GPTQ-Int4-sym-G128-MTP-BF16-MixedCal-v2	36B	24.1 GB	Too big
Qwen3.6-35B-A3B-uncensored-heretic-Native-MTP-Preserved	36B	24.1 GB	Too big
Qwen3.6-35B-A3B	36B	24.1 GB	Too big
SIQ-1-35B-B70-Turbo	36B	24.1 GB	Too big
Qwen3.6-35B-A3B-abliterated-v4	36B	24.1 GB	Too big
Kimi-Linear-48B-A3B-Instruct	49B	32.5 GB	Too big
Llama-3.3-70B-Instruct	71B	46.7 GB	Too big
Qwen3-Coder-Next	80B	52.5 GB	Too big
Llama-4-Scout-17B-16E-Instruct	109B	71.3 GB	Too big
gpt-oss-120b	117B	76.5 GB	Too big
Laguna-S-2.1	118B	77.1 GB	Too big
NVIDIA-Nemotron-3.5-Super-VL-120B-A12B	120B	78.4 GB	Too big
Qwen 3.8 Flash	125B	81.7 GB	Too big
Qwen 3.8 Flash-Next Base	125B	81.7 GB	Too big
Ling-3.0-flash	127B	83.0 GB	Too big
DeepSeek-V4-Flash-180B	158B	103.0 GB	Too big
Qwen3.8-Flash-Next	180B	117.2 GB	Too big
Qwen3.8-Flash-Next-REAP-256-duo	180B	117.2 GB	Too big
Qwen3.8-Flash-Next-hibrid48	180B	117.2 GB	Too big
CYBER-FROST-3.8	180B	117.2 GB	Too big
MiniMax-M2.7	229B	148.9 GB	Too big
Baichuan-M3-235B	235B	152.8 GB	Too big
DeepSeek-V4-Flash	291B	189.0 GB	Too big
Hy3	299B	194.2 GB	Too big
DeepSeek-V4-Flash-0731	304B	197.4 GB	Too big
DeepSeek-V4-Flash-0731-ROCmFP3	304B	197.4 GB	Too big
DeepSeek-V4-Flash-Vision-Exp	305B	198.1 GB	Too big
keys-DeepSeekV4Flash-Vision-EXP-ablit	305B	198.1 GB	Too big
GLM-5.3-Flash	321B	208.4 GB	Too big
DeepSeek-V4.1-Flash	552B	357.9 GB	Too big
DeepSeek-V4.1-Flash-1017	552B	357.9 GB	Too big
GLM-5.2	753B	487.9 GB	Too big

How this is estimated: weights = parameters × bits per weight; we add ~15% for KV cache at moderate context and runtime overhead, plus 0.8 GB engine floor. Long contexts, larger batch sizes and vision towers raise the real number — treat a "Tight" as "possible with reduced context". Measured throughput lives on the benchmark board.