Qwen3.8-27B

Qwen family · 28.0B params

67recorded runs

4546.8best tok/s

44verified

Evidence: 44 sourced runs · 23 awaiting review · 3 independent sources · first recorded Aug 14, 2026 · last recorded Sep 29, 2026. What these labels mean.

What the Qwen3.8-27B numbers mean

The fastest verified result on our board is 435.1 output tok/s on an RTX 5090 at IQ4_XS with a 131072-token context. That is about 2.3 ms per generated token — 500 tokens takes roughly 1.1 seconds. Prompt processing on the same record runs at 1030.5 tok/s, so a 2,000-token prompt is read in about 1.9 seconds. Peak VRAM was measured at 24.5 GB. Across the 67 records we hold for this model the range is 16.8–4546.8 tok/s on 31 GPUs, because quantisation, context length and the engine build all move the number.

How much VRAM does Qwen3.8-27B need?

Qwen3.8-27B has 28.0B parameters. These estimates use the same formula as the VRAM calculator: weights, plus about 15% for a moderate-context KV cache and runtime overhead, plus a small engine floor.

Quantisation	Estimated VRAM	Smallest card that fits
Q4_K_M (4-bit)	18.9 GB	24 GB
Q8_0 (8-bit)	35.0 GB	48 GB
FP16 (16-bit)	65.2 GB	80 GB

Long contexts and large batches raise the real number — read the smallest card as "possible with a reduced context". Measured throughput is on the benchmark board.

Fastest GPUs for this model (best tok/s)

Models with a similar footprint

Model	Params	Q4_K_M weights
Bonsai-27B	28B	15.8 GB
Godoter-27B	28B	15.8 GB
Huihui-Qwen3.8-27B-abliterated	28B	15.8 GB
Qwen3.5-27B	28B	15.8 GB
Qwen3.6-27B	28B	15.8 GB
Qwen3.6-27B-i1	28B	15.8 GB

Ranked by distance from Qwen3.8-27B's parameter count. Full benchmark board.

All benchmarks Methodology

GPU	Model	Quant	Backend	Ctx	tok/s	prompt tok/s	VRAM GB	Source
RTX 5090 · 32 GB	Qwen3.8-27B	IQ4_XS	lucebox 298031aa4222ec61c971ed834ec8f8829ce3…	131072	435.1 ✓	1030.5	24.5	src
Tesla V100 x4 · 16 GB	Qwen3.8-27B	NVFP4	vllm 1.2.2	32768	391.0 ✓	1185.0	61.0	src
RTX PRO 6000 Blackwell · 96 GB	Qwen3.8-27B	NVFP4	sglang dev-qwen38-27b-dflash2	262144	296.0 ✓	—	—	src
CMP 170HX · 64 GB	Qwen3.8-27B	W4A16	vllm 0.27.1	4359	255.9 ✓	2321.6	—	src
RTX 3090 x2 · 24 GB	Qwen3.8-27B	AutoRound INT4 + fp8 KV	sglang 0.5.19	262000	219.8 ✓	—	—	src
RTX 5090 · 32 GB	Qwen3.8-27B	NVFP4	tensorrt-llm NInfer 6e2786c5e5a58c2a8804595a9042f…	16384	213.0 ✓	9125.8	23.2	src
RTX 5090 x2 · 32 GB	Qwen3.8-27B	NVFP4	sglang 0.0.0.dev331+g0149f56e8 (main@0149f5…	262144	195.6 ✓	16851.3	57.2	src
RTX 3090 x2 · 24 GB	Qwen3.8-27B	W4A16-AutoRound	vllm 0.27.1+syv-ai.00210159	262144	182.4 ✓	1338.2	—	src
CMP 170HX · 64 GB	Qwen3.8-27B	BF16	vllm 0.27.1	4352	181.6 ✓	—	—	src
RTX 3090 · 24 GB	Qwen3.8-27B	EXL3-3.5bpw	exllamav3 1.4.2 (Mia fork 63b32f001d7b2cfed3b3…	24576	164.6 ✓	586.9	17.3	src
RX 7900 XTX · 24 GB	Qwen3.8-27B	MQ4	hipfire 80a572c	2048	154.2 ✓	51.3	17.1	src
RTX 3090 · 24 GB	Qwen3.8-27B	EXL3 4.00bpw	exllamav2 v1.5.0	8192	152.4 ✓	—	—	src
RTX 3090 · 24 GB	Qwen3.8-27B	EXL3 4.00bpw	buun-llama-cpp 1d0f493c73817176f6953069b7c10211de95…	262144	149.2 ✓	—	—	src
CMP 170HX · 64 GB	Qwen3.8-27B	INT8	vllm 0.27.1	65536	147.7 ✓	—	—	src
RTX 4090 · 24 GB	Qwen3.8-27B	int4 (output head only)	vllm pinned version (exact version not sp…	38000	141.2 ✓	—	—	src
CMP 170HX · 64 GB	Qwen3.8-27B	W4A16	vllm 0.27.1	65536	140.5 ✓	—	—	src
RTX 5090 · 32 GB	Qwen3.8-27B	W4A16	vllm cu129-nightly-e9d1398d9 (native DFla…	2048	137.9 ✓	—	—	src
RTX 3090 · 24 GB	Qwen3.8-27B	AWQ-INT4 W4A16	sglang 2e7c85da6	8000	136.8 ✓	1616.0	—	src
M4 Max 128GB	Qwen3.8-27B	4-bit	mlx 0.3.4 → 0.3.6.3	48800	134.0 ✓	—	—	src
Instinct MI100 x4 · 32 GB	Qwen3.8-27B	INT8	vllm https://github.com/curvedinf/vllm-gf…	64000	132.4 ✓	1202.7	128.0	src
RTX 5090 · 32 GB	Qwen3.8-27B	W4A16	vllm v0.27.1	2048	108.0 ✓	—	—	src
Intel Arc Pro B70 x2 · 32 GB	Qwen3.8-27B	AutoRound INT4 W4A16	vllm v0.20.2rc1.dev13+g9557d9108 local XP…	2048	101.9 ✓	—	—	src
RTX PRO 6000 Blackwell · 96 GB	Qwen3.8-27B	FP8	vllm 0.26.0	131072	101.4 ✓	—	—	src
RTX 3090 Ti · 24 GB	Qwen3.8-27B	IQ4_XS	llama.cpp custom	204800	97.0 ✓	1471.0	17.3	src
RTX 4090 · 24 GB	Qwen3.8-27B	UD-Q4_K_XL	llama.cpp llama.cpp 7e4c0a96880dae4fc4268ad441…	172032	91.6 ✓	968.5	—	src
RTX PRO 6000 Blackwell · 96 GB	Qwen3.8-27B	Q8_K_XL	llama.cpp 0.1.2-dev build 10498	99100	88.1 ✓	2311.4	—	src
RTX 3090 · 24 GB	Qwen3.8-27B	Q4_K_XL	llama.cpp 0.3.0-dev (build 1, commit d7bd3bf)	24576	81.9 ✓	517.1	18.9	src
RTX 5090 · 32 GB	Qwen3.8-27B	UD-Q6_K	llama.cpp b10478-adcf720f3	251438	79.4 ✓	541.0	30.3	src
RTX 4090 D · 24 GB	Qwen3.8-27B	Q4_K_M	llama.cpp b358-7620399	2421	70.9 ✓	1467.0	23.5	src
Intel Arc Pro B70 x2 · 32 GB	Qwen3.8-27B	INT4_W4A16	vllm v0.27.2rc1.dev78+g8e6d8e4f6	2048	67.7 ✓	770.0	29.0	src
RTX 4090 D · 24 GB	Qwen3.8-27B	IQ3_S (GSQ-RCO)	llama.cpp c56093 (AtomicBot turboquant fork)	262144	67.5 ✓	—	20.7	src
Radeon AI Pro R9700 · 32 GB	Qwen3.8-27B	Q4_0	llama.cpp b10711	262144	64.3 ✓	—	—	src
RX 7900 XTX · 24 GB	Qwen3.8-27B	Q4_K_M	llama.cpp b10791	2052	58.0 ✓	921.0	22.9	src
DGX Spark · 128 GB	Qwen3.8-27B	NVFP4 W4A4	sglang lmsysorg/sglang:dev-qwen38-27b-dflas…	262144	56.0 ✓	—	—	src
DGX Spark · 128 GB	Qwen3.8-27B	NVFP4 W4A4	sglang lmsysorg/sglang:qwen38-27b	262144	51.5 ✓	—	—	src
RTX 5070 · 12 GB	Qwen3.8-27B	UD-Q2_K_XL	llama.cpp b10449	32768	48.3 ✓	912.3	9.0	src
RTX 3080 · 10 GB	Qwen3.8-27B	IQ2_XXS	llama.cpp b10472	65536	44.4 ✓	1079.4	9.7	src
Radeon AI Pro R9700 x2 · 32 GB	Qwen3.8-27B	Q8_K_XL	llama.cpp 5ea1b124e + stew675 01/10/02	262144	43.6 ✓	1191.0	—	src
RX 7800 XT · 16 GB	Qwen3.8-27B	Unsloth-Dynamic-IQ4_XS	llama.cpp b10622	121000	42.0 ✓	500.0	—	src
RTX 3090 · 24 GB	Qwen3.8-27B	EXL3 4.00bpw	buun-llama-cpp 1d0f493c73817176f6953069b7c10211de95…	150000	27.8 ✓	525.3	—	src
Intel Arc Pro B70 · 32 GB	Qwen3.8-27B	Q4_K_M	llama.cpp 4302fb599 + neural.download Qwen3.8 …	8192	27.8 ✓	—	—	src
RTX 3090 · 24 GB	Qwen3.8-27B	EXL3 4.00bpw	exllamav2 v1.5.0	150000	24.5 ✓	—	—	src
Tesla V100 x2 · 32 GB	Qwen3.8-27B	Q8_K_XL	llama.cpp 9463	131702	24.0 ✓	1200.0	24.0	src
RTX 5090 · 32 GB	Qwen3.8-27B	FP8	vllm 0.27.1	8192	16.8 ✓	—	30.9	src
RTX 5090 · 32 GB	Qwen3.8-27B	NVFP4	sglang —	2048	4546.8 …	—	29.8	src
CMP 170HX · 64 GB	Qwen3.8-27B	W4A16	sglang 0.5.19	1280	1392.4 …	—	—	src
H100 NVL x2 · 94 GB	Qwen3.8-27B	FP8	sglang —	2048	293.3 …	1080.5	—	src
RTX PRO 6000 Blackwell · 96 GB	Qwen3.8-27B	NVFP4	llama.cpp —	2048	120.6 …	351.0	—	src
RTX 3090 · 24 GB	Qwen3.8-27B	W4A16	vllm —	65536	113.2 …	—	22.4	src
RTX PRO 6000 Blackwell · 96 GB	Qwen3.8-27B	NVFP4	sglang —	2048	97.4 …	1828.6	—	src
RTX 3090 x2 · 24 GB	Qwen3.8-27B	GPTQ-4bit	vllm —	8192	96.6 …	1311.1	45.3	src
Multi-GPU x2 · 24 GB	Qwen3.8-27B	Q4_K_M	llama.cpp —	262144	86.3 …	—	28.0	src
Intel Arc Pro B70 x2 · 32 GB	Qwen3.8-27B	FP8	vllm 0.27.2rc1.dev77+gac7509e2b; XPU kern…	1024	82.4 …	—	—	src
RTX 5060 Ti x2 · 16 GB	Qwen3.8-27B	NVFP4	vllm —	2048	63.2 …	395.5	—	src
RTX 3060 x2 · 12 GB	Qwen3.8-27B	UD-Q4_K_XL	llama.cpp —	65536	57.0 …	—	—	src
RTX 5070 · 12 GB	Qwen3.8-27B	EXL3 2.20bpw (SC_2.20bpw_H3_V3)	exllamav3 1.4.6+cu128	126976	50.2 …	754.5	10.4	src
Intel Arc Pro B70 · 32 GB	Qwen3.8-27B	GPTQ-4bit	vllm —	2048	44.0 …	806.7	—	src
RTX 4060 Ti x2 · 16 GB	Qwen3.8-27B	UD-Q6_K_XL	llama.cpp —	70000	43.6 …	934.5	—	src
RTX 3060 · 12 GB	Qwen3.8-27B	IQ2_XS-MTP	llama.cpp GSQ-RCO+Unsloth MTP	82000	38.0 …	460.0	—	src
RTX 4070 · 12 GB	Qwen3.8-27B	UD-IQ3_XXS	llama.cpp —	32768	36.5 …	65.2	10.8	src
A100 · 40 GB	Qwen3.8-27B	Q8_0	llama.cpp —	65536	33.7 …	1088.2	30.0	src
CMP 170HX · 64 GB	Qwen3.8-27B	UD-Q4_K_XL	llama.cpp 0.3.0-dev+b10705	131295	29.7 …	595.7	—	src
RX 7800 XT · 16 GB	Qwen3.8-27B	Unsloth-Dynamic-IQ4_XS	llama.cpp —	2048	28.0 …	500.0	—	src
CMP 170HX · 64 GB	Qwen3.8-27B	UD-Q8_K_XL	llama.cpp 0.3.0-dev+b10705	131295	25.9 …	610.2	—	src
RTX 3060 · 12 GB	Qwen3.8-27B	IQ2_XS	llama.cpp GSQ-RCO	158000	22.0 …	460.0	—	src
Intel Arc Pro B60 · 24 GB	Qwen3.8-27B	Q4_K_M	llama.cpp —	2048	21.2 …	504.4	—	src
RTX 4070 · 12 GB	Qwen3.8-27B	GSQ-RCO-IQ2_XS	llama.cpp —	152000	17.6 …	315.0	11.5	src