Run Qwen3.8-27B on a Single RTX 3060 (12GB)

As of 2026-09-05. Community-sourced run by @DogukanUrker — attributed, and not re-run by us. Numbers are the poster's measurements.

TL;DR

A 27B model on a $495-class 12GB card (an RTX 3060, 2026-09-28 snapshot), with 158K tokens of usable context. Two-bit quantization is how it fits; an MTP draft head is how it gets fast.

Why this is interesting

Conventional advice says 27B-class models need 24GB at Q4. This run goes the other direction: IQ2_XS (2-bit) quant + KV-cache quant + MTP draft, on a 12GB RTX 3060. You trade quality for context and speed — and for long single-session agent runs, context wins.

The author tested four IQ2 quants across seven benchmarks and GSQ-RCO IQ2_XS won while being the second-smallest file of the four. The catch: GSQ-RCO ships no MTP head, so you borrow Unsloth's (same base model) and pay ~76K of context for 1.7× decode.

The two builds (llama-server)

Max context (158K, 22 tok/s):

llama-server -m Qwen3.8-27B-GSQ-RCO-IQ2_XS.gguf -ngl 99 -c 158000 -fa on --jinja -np 1 --cache-type-k q4_0 --cache-type-v q4_0 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --presence-penalty 0.0 --repeat-penalty 1.0 --reasoning on --chat-template-kwargs '{"reasoning_effort":"low"}'

MTP draft (82K, 38 tok/s):

llama-server -m Qwen3.8-27B-GSQ-RCO-IQ2_XS.gguf -md mtp-Qwen3.8-27B-Q4_0.gguf -ngl 99 -c 82000 -fa on --jinja -np 1 --spec-type draft-mtp --cache-type-k q4_0 --cache-type-v q4_0 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --presence-penalty 0.0 --repeat-penalty 1.0 --reasoning on --chat-template-kwargs '{"reasoning_effort":"low"}'

Key flags: --cache-type-k q4_0 --cache-type-v q4_0 quantize the KV cache (that is how 158K fits in 12GB), --spec-type draft-mtp adds the draft head for the speed build, and reasoning_effort=low keeps 2-bit reasoning from spiraling.

Which one to pick

• Short few-shot tasks (never touch 64K): the MTP build — extra context is dead weight, take the speed.
• Long single-session agent runs (blow past 82K): the 158K build — no decode rate saves you once you are out of context.

Both are now on our board as attributed import rows: benchmarks · compatibility matrix.

For buyers: the 12GB tier is not dead

This is the strongest recent evidence that the 12GB tier still has a place in 2026 — an RTX 3060 12GB (~$495, 2026-09-28 snapshot) running a 27B at useful context. Check the value index for where 12GB sits in $/GB, and the quantization guide for the IQ2 vs Q4 tradeoff.

Sources

• @DogukanUrker — Qwen3.8-27B on a single RTX 3060 (2026-09-05)
• llama.cpp
• DiggTop data: benchmarks · matrix · value

Community-sourced; the rows state engine, quant and context, but DiggTop did not run them.