Qwen3.8-27B Uncensored: MTP Kept, Cost Measured

As of 2026-09-07. Community release by Jonathan Coletti (JonathanColetti/Qwen3.8-27B-Uncensored-GGUF, Apache 2.0). All measurements below are the author's, quoted from the model card — attributed, and not re-run by us. The release made the rounds via @TeksEdge.

Most "uncensored" releases are weight edits with no accounting: you don't know what the surgery cost, and many drop the MTP head in the process because speculative decoding is the first thing to break. This one does the opposite of both. Coletti abliterated the official Qwen/Qwen3.8-27B with Heretic (which trades refusal count against KL divergence from base), grafted all 15 MTP tensors back verbatim from the base checkpoint, and published the numbers for what it all cost: refusals, capability, perplexity per quant, and measured MTP speedups.

Why a hardware site covers it: this is a 27B dense model with vision and a 262K context architecture, and the 2-bit GGUF is 10.6 GB — squarely in the 16GB-card territory we benchmark. A 10.6 GB dense 27B with a working MTP head changes what a $200 card can do.

How it was made

• Heretic abliteration at bf16 — no finetuning, no extra data, no handwritten refusal-removal code. It modifies attn.o_proj and mlp.down_proj in the main stack; the resulting LoRA is merged into the bf16 base, so the published weights are not a quantized round trip.
• MTP head copied verbatim from the base checkpoint after merging — abliteration never touches it. The catch: the draft head was trained against the unmodified model, so acceptance rate may fall slightly. Speculative decoding verifies every token against the target, so output quality is unaffected either way.
• iMatrix computed from the f16 directly (wikitext-2 raw, 200 chunks, published as a file), so every quant in the repo was calibrated against real weights, not an intermediate quant.

What the surgery cost

Against the unmodified base, identical settings, 0-shot via lm-evaluation-harness, both models in the same session:

Every delta sits within or near the reported standard error (e.g. MMLU ±0.30, ARC ±1.44), so none is clearly separable from run-to-run noise. The author is explicit that these are 0-shot scores, not comparable to Qwen's few-shot published numbers — they're comparable to each other, which is the point.

Two reading notes from the card: the refusal figure is not an over-refusal rate — it measures how much of the original refusal behaviour on harmful requests remains, in non-thinking mode. And the published model is one point on a Heretic Pareto front (200 trials, 23 non-dominated points): more refusal reduction is available at the cost of higher KL (e.g. 44/100 refusals at KL 0.0352).

The weight edit itself costs 0.0434 perplexity on wikitext-2 (6.5129 → 6.5563, ~0.7% relative). Measured as a paired difference over the same tokens: 0.0437 ± 0.0163 — a real cost, not noise, with the base Q8_0 as control (0.0007 from base bf16).

What the quantization costs

The repo ships fused MTP files (MTP rides inline as a built-in draft), noMTP twins, a standalone MTP draft head (Q8_0 / Q4_0), and an F16 vision projector. Converted with llama.cpp a94d563ed, all from the same published iMatrix:

The author's own caveat, worth repeating: every row except IQ2_M sits inside a span of 0.026 against a standard error of ~0.25 — Q8_0 is not worse than Q5_K_M, that ordering is noise. The only difference the measurement resolves is IQ2_M, ~2.8 standard errors above baseline. (Perplexity detects gross quant damage and nothing else — no reasoning, code, or multilingual signal.)

What MTP actually buys (measured on this build)

Original-release table, hardware not recorded — compare the ratios:

Past n_max ~3–4 the gain collapses — acceptance falls as the draft head, trained on the unmodified model, meets the edited weights. Code benefits most (1.28× at n_max 1); by n_max 5–8 every prompt type is slower than the no-spec baseline.

Measured on an NVIDIA H200 NVL, IQ2_M, 256 generated tokens, median of 3:

The MTP head survives 2-bit quantization because it's pinned to q8_0 rather than quantized with the rest of the stack. And a nice detail: pairing noMTP-IQ2_M with the standalone draft-Q8_0 hits 97.7 tok/s on prose at n_max 2 (1.30×) — better than the fused file on that prompt, because the fused head shares the low-bit token_embd/output tensors with the main model while the standalone draft carries its own Q8_0 copies. Costs 13.3 GB of weights vs 10.6 GB, so it only wins if you have the VRAM to spare.

What this means for your hardware

• 16GB cards: the IQ4_XS (15.3 GB) and Q4_K_M (16.8 GB) files are the natural fits, IQ2_M (10.6 GB) the headroom play. File size ≠ VRAM — a 15.3 GB file does not imply comfortable 16GB operation with useful context; you still need KV cache and compute buffers on top. See our VRAM cliff explainer for the budgeting math, and our existing Qwen3.8-27B on a 12GB RTX 3060 guide for what a 12GB card already does with this model family.
• Third-party ladder: @TeksEdge points to iMatrix quantizations by mradermacher (IQ3_S 12.7 GB, IQ3_M 12.9 GB, Q3_K_M 13.6 GB, IQ4_XS 15.4 GB, Q4_K_M 16.9 GB) — the ~13 GB bracket is the sweet spot for 16GB cards if those files check out. The model card itself links one third-party set (zerodigest/YMQ-MTP) explicitly not as an endorsement.
• llama.cpp version matters: MTP speculative decoding for this architecture landed in llama.cpp PR #22673. Older builds load these files and silently ignore the MTP tensors — you'd be running the slower model with no error.
• Evaluate at Q6_K or Q8_0, not IQ2_M: the card's own guidance is that refusal-boundary behaviour is already less stable than base after abliteration, and lower quants compound it. Nothing measures the refusal boundary at 2-bit.

Caveats the author states

• Refusals are reduced, not eliminated (12/100 still refuse) and not redirected; behaviour near the old refusal boundary is less stable than base.
• The refusal test is 100 prompts from one dataset (harmful requests) — it generalizes to that distribution and no further.
• Capability coverage stops at 4 0-shot benchmarks: no generative, math, code, or multilingual evaluation was run, and the harness loads the text stack only — nothing measures the vision tower.
• All behaviour measurements were on the bf16 merge; the files you download are quantized, and quantization compounds everything above.
• These are the poster's measurements, not ours — the "uncensored = capability unchanged" claim holds only within the tested set.

If you run it on your card, the tok/s belongs on the board — submit your run with engine, quant, and context, and it lands on the board with your name on it.

Sources

• JonathanColetti/Qwen3.8-27B-Uncensored-GGUF — model card (method, files, PPL, behaviour and speculative-decoding tables, caveats)
• JonathanColetti/Qwen3.8-27B-Uncensored — bf16 base repo
• @TeksEdge on X (release summary; mradermacher iMatrix sizes as reported there)
• Qwen/Qwen3.8-27B — base checkpoint