Qwen3.8-Flash-Next
40recorded runs
204.5best tok/s
38verified
What the Qwen3.8-Flash-Next numbers mean
The fastest verified result on our board is 186.9 output tok/s on an CMP 170HX x2 at W4A16-FP8PLE with a 262052-token context. That is about 5.3 ms per generated token — 500 tokens takes roughly 2.7 seconds. Across the 40 records we hold for this model the range is 20.3–204.5 tok/s on 16 GPUs, because quantisation, context length and the engine build all move the number.
How much VRAM does Qwen3.8-Flash-Next need?
Qwen3.8-Flash-Next has 180.0B parameters. These estimates use the same formula as the VRAM calculator: weights, plus about 15% for a moderate-context KV cache and runtime overhead, plus a small engine floor.
| Quantisation | Estimated VRAM | Smallest card that fits |
|---|---|---|
| Q4_K_M (4-bit) | 117.2 GB | 141 GB |
| Q8_0 (8-bit) | 220.7 GB | multi-GPU or CPU offload |
| FP16 (16-bit) | 414.8 GB | multi-GPU or CPU offload |
Fastest GPUs for this model (best tok/s)
Models with a similar footprint
| Model | Params | Q4_K_M weights |
|---|---|---|
| CYBER-FROST-3.8 | 180B | 101.2 GB |
| Qwen3.8-Flash-Next-hibrid48 | 180B | 101.2 GB |
| Qwen3.8-Flash-Next-REAP-256-duo | 180B | 101.2 GB |
| DeepSeek-V4-Flash-180B | 158B | 88.9 GB |
| MiniMax-M2.7 | 229B | 128.8 GB |
| Ling-3.0-flash | 127B | 71.4 GB |
| GPU | Model | Quant | Backend | Ctx | tok/s | prompt tok/s | VRAM GB | Source |
|---|---|---|---|---|---|---|---|---|
| CMP 170HX x2 · 64 GB | Qwen3.8-Flash-Next | W4A16-FP8PLE | vllm 0.29.0 | 262052 | 186.9 ✓ | — | — | src |
| RTX 3090 x4 · 24 GB | Qwen3.8-Flash-Next | W4A16+FP8-KV | vllm tonyd2wild qwen4exp-ple (vllm nightl… | 262144 | 145.8 ✓ | — | — | src |
| Mac Studio (Mac16,9) with Apple M4 Max · 64 GB | Qwen3.8-Flash-Next | IQ3_XXS + MTP Q8_0 | llama.cpp 781dbc5ac98921dbdb5e5b2ec5b7a50960e9… | 1024 | 110.3 ✓ | — | — | src |
| Tesla V100 x2 · 32 GB | Qwen3.8-Flash-Next | IQ3XXS | llama.cpp b11030-mix-5ff778e | 4000 | 96.1 ✓ | — | — | src |
| DGX Spark x2 · 128 GB | Qwen3.8-Flash-Next | INT4-AutoRound | vllm 0.30.0 with patches (v5.2 image, sha… | 128000 | 93.0 ✓ | — | — | src |
| Tesla V100 x2 · 32 GB | Qwen3.8-Flash-Next | IQ3XXS | llama.cpp b11030-mix-5ff778e | 512 | 87.5 ✓ | — | — | src |
| RTX 5070 · 12 GB | Qwen3.8-Flash-Next | Q2_0 | strata 0.1.26 | 32768 | 81.8 ✓ | — | — | src |
| Tesla V100 x2 · 32 GB | Qwen3.8-Flash-Next | IQ3XXS | llama.cpp b11030-mix-5ff778e | 131000 | 80.9 ✓ | — | — | src |
| DGX Spark x2 · 128 GB | Qwen3.8-Flash-Next | NVFP4 | vllm vLLM + patches from github.com/bilik… | 262144 | 80.0 ✓ | — | — | src |
| DGX Spark · 128 GB | Qwen3.8-Flash-Next | 4.05bpw_h6_ng6 | tensorfold 0.6.5 | 40960 | 75.7 ✓ | — | — | src |
| DGX Spark · 128 GB | Qwen3.8-Flash-Next | EXL3 | exllamav2 523ecd3 | 240000 | 72.0 ✓ | — | — | src |
| DGX Spark · 128 GB | Qwen3.8-Flash-Next | EXL3 | exllamav2 523ecd3 | 4000 | 69.8 ✓ | — | — | src |
| DGX Spark · 128 GB | Qwen3.8-Flash-Next | EXL3 | exllamav2 523ecd3 | 128000 | 69.5 ✓ | — | — | src |
| DGX Spark · 128 GB | Qwen3.8-Flash-Next | EXL3 3.05bpw | exllamav2 523ecd3 | 75000 | 67.6 ✓ | — | — | src |
| DGX Spark · 128 GB | Qwen3.8-Flash-Next | EXL3 3.05bpw | exllamav2 523ecd3 | 32000 | 59.0 ✓ | — | — | src |
| DGX Spark · 128 GB | Qwen3.8-Flash-Next | EXL3 3.05bpw | exllamav2 523ecd3 | 1000 | 56.0 ✓ | — | — | src |
| DGX Spark · 128 GB | Qwen3.8-Flash-Next | NVFP4 | sglang 8874c51a | 262144 | 55.0 ✓ | — | — | src |
| CMP 170HX · 64 GB | Qwen3.8-Flash-Next | UD-IQ1_S | llama.cpp pr-27742 | 65536 | 46.1 ✓ | 68.7 | 43.2 | src |
| DGX Spark · 128 GB | Qwen3.8-Flash-Next | NVFP4+FP8 hybrid | vllm blazux/qwen3.8-Flash-DGX (vllm fork)… | 258790 | 44.2 ✓ | 1636.7 | — | src |
| DGX Spark · 128 GB | Qwen3.8-Flash-Next | NVFP4 | vllm 8a728663 | 262144 | 43.9 ✓ | — | — | src |
| DGX Spark · 128 GB | Qwen3.8-Flash-Next | NVFP4+FP8 hybrid | vllm blazux/qwen3.8-Flash-DGX (vllm fork)… | 4292 | 43.0 ✓ | 1417.3 | — | src |
| DGX Spark · 128 GB | Qwen3.8-Flash-Next | NVFP4+FP8 hybrid | vllm blazux/qwen3.8-Flash-DGX (vllm fork)… | 32850 | 42.6 ✓ | 1491.4 | — | src |
| DGX Spark · 128 GB | Qwen3.8-Flash-Next | NVFP4+FP8 hybrid | vllm blazux/qwen3.8-Flash-DGX (vllm fork)… | 16424 | 42.3 ✓ | 1322.3 | — | src |
| DGX Spark · 128 GB | Qwen3.8-Flash-Next | NVFP4+FP8 hybrid | vllm blazux/qwen3.8-Flash-DGX (vllm fork)… | 8232 | 42.1 ✓ | 1414.9 | — | src |
| DGX Spark · 128 GB | Qwen3.8-Flash-Next | NVFP4+FP8 hybrid | vllm blazux/qwen3.8-Flash-DGX (vllm fork)… | 864 | 41.6 ✓ | 580.6 | — | src |
| DGX Spark · 128 GB | Qwen3.8-Flash-Next | NVFP4+FP8 hybrid | vllm blazux/qwen3.8-Flash-DGX (vllm fork)… | 66177 | 41.4 ✓ | 1780.4 | — | src |
| DGX Spark · 128 GB | Qwen3.8-Flash-Next | NVFP4+FP8 hybrid | vllm blazux/qwen3.8-Flash-DGX (vllm fork)… | 2271 | 41.1 ✓ | 1132.0 | — | src |
| Tesla V100 PCIe 16GB · 16 GB | Qwen3.8-Flash-Next | Q2_0 | strata v0.1.42 | 8192 | 40.7 ✓ | — | — | src |
| DGX Spark · 128 GB | Qwen3.8-Flash-Next | NVFP4+FP8 hybrid | vllm blazux/qwen3.8-Flash-DGX (vllm fork)… | 131437 | 40.6 ✓ | 1718.3 | — | src |
| RTX 3090 x2 · 24 GB | Qwen3.8-Flash-Next | Unsloth-Dynamic-IQ3_XXS | llama.cpp 0.3.0-dev / commit 035e227 (llamacpp… | 217088 | 40.1 ✓ | 340.1 | 42.9 | src |
| RTX 3090 · 24 GB | Qwen3.8-Flash-Next | EXL3 2.50bpw | exllamav2 v1.5.0-compatible build with MoE CPU… | 262144 | 38.6 ✓ | — | — | src |
| DGX Spark · 128 GB | Qwen3.8-Flash-Next | NVFP4+FP8 hybrid | vllm blazux/qwen3.8-Flash-DGX (vllm fork)… | 327 | 37.4 ✓ | 132.0 | — | src |
| Radeon AI Pro R9700 x3 · 32 GB | Qwen3.8-Flash-Next | UD-IQ3_XXS | llama.cpp b10643-11-gaecd368d0 | 829 | 28.1 ✓ | 474.4 | — | src |
| Radeon AI Pro R9700 · 32 GB | Qwen3.8-Flash-Next | IQ4_XS | llama.cpp 035e227 + Qwen4Exp graph-reuse/QSA t… | 32768 | 25.6 ✓ | 537.4 | 32.9 | src |
| Intel Arc Pro B70 · 32 GB | Qwen3.8-Flash-Next | EXL3 3.05 bpw | sglang 24c872759256 | 8192 | 24.0 ✓ | 1056.0 | — | src |
| Intel Arc Pro B70 · 32 GB | Qwen3.8-Flash-Next | EXL3 3.05 bpw | sglang 24c872759256 | 32768 | 23.9 ✓ | 1106.0 | — | src |
| RTX 3090 · 24 GB | Qwen3.8-Flash-Next | EXL3 2.50bpw | exllamav2 v1.5.0-compatible build with MoE CPU… | 175000 | 20.9 ✓ | — | — | src |
| Intel Arc Pro B70 x2 · 32 GB | Qwen3.8-Flash-Next | IQ3_S | llama.cpp 0.3.0-dev b492 9723942 SYCL F16=ON | 16384 | 20.3 ✓ | 594.5 | — | src |
| RTX PRO 6000 Blackwell x2 · 192 GB | Qwen3.8-Flash-Next | FP8_E4M3 | vllm — | 262144 | 204.5 … | — | — | src |
| RTX PRO 6000 Blackwell x2 · 192 GB | Qwen3.8-Flash-Next | NVFP4 | vllm — | 262144 | 93.5 … | 437.7 | — | src |