NVIDIA-Nemotron-Nano-9B-v2

Nemotron family · 9.0B params

1recorded run

64.2best tok/s

1verified

Evidence: 1 sourced run · 1 independent source · first recorded Jul 2, 2026. What these labels mean.

What the NVIDIA-Nemotron-Nano-9B-v2 numbers mean

The fastest verified result on our board is 64.2 output tok/s on an RTX 3060 Ti at IQ4_XS with a 2048-token context. That is about 15.6 ms per generated token — 500 tokens takes roughly 7.8 seconds. Prompt processing on the same record runs at 1755.9 tok/s, so a 2,000-token prompt is read in about 1.1 seconds.

How much VRAM does NVIDIA-Nemotron-Nano-9B-v2 need?

NVIDIA-Nemotron-Nano-9B-v2 has 9.0B parameters. These estimates use the same formula as the VRAM calculator: weights, plus about 15% for a moderate-context KV cache and runtime overhead, plus a small engine floor.

Quantisation	Estimated VRAM	Smallest card that fits
Q4_K_M (4-bit)	6.6 GB	8 GB
Q8_0 (8-bit)	11.8 GB	16 GB
FP16 (16-bit)	21.5 GB	32 GB

Long contexts and large batches raise the real number — read the smallest card as "possible with a reduced context". Measured throughput is on the benchmark board.

Fastest GPUs for this model (best tok/s)

RTX 3060 Ti64.2

Models with a similar footprint

Model	Params	Q4_K_M weights
Carnice-9b	9B	5.1 GB
DeepSeek-V4-Pro-Qwen3.5-9B-MTP	9B	5.1 GB
Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated-i1	9B	5.1 GB
Ornith-1.0-9B	9B	5.1 GB
Qwen3-VL-8B-Instruct	9B	5.1 GB
Qwen3.5-9B-MTP	9B	5.1 GB

Ranked by distance from NVIDIA-Nemotron-Nano-9B-v2's parameter count. Full benchmark board.

All benchmarks Methodology

GPU	Model	Quant	Backend	Ctx	tok/s	prompt tok/s	VRAM GB	Source
RTX 3060 Ti · 8 GB	NVIDIA-Nemotron-Nano-9B-v2	IQ4_XS	llama.cpp llama.cpp b9860	2048	64.2 ✓	1755.9	—	src