Hy3
1recorded run
27.6best tok/s
1verified
What the Hy3 numbers mean
The fastest verified result on our board is 27.6 output tok/s on an CMP 170HX x2 at Q2_K_XL with a 32768-token context. That is about 36.2 ms per generated token — 500 tokens takes roughly 18.1 seconds. Prompt processing on the same record runs at 250.7 tok/s, so a 2,000-token prompt is read in about 8.0 seconds. Peak VRAM was measured at 115.9 GB.
How much VRAM does Hy3 need?
Hy3 has 299.0B parameters. These estimates use the same formula as the VRAM calculator: weights, plus about 15% for a moderate-context KV cache and runtime overhead, plus a small engine floor.
| Quantisation | Estimated VRAM | Smallest card that fits |
|---|---|---|
| Q4_K_M (4-bit) | 194.2 GB | multi-GPU or CPU offload |
| Q8_0 (8-bit) | 366.1 GB | multi-GPU or CPU offload |
| FP16 (16-bit) | 688.5 GB | multi-GPU or CPU offload |
Fastest GPUs for this model (best tok/s)
- CMP 170HX x227.6
Models with a similar footprint
| Model | Params | Q4_K_M weights |
|---|---|---|
| DeepSeek-V4-Flash-0731 | 304B | 171.0 GB |
| DeepSeek-V4-Flash-0731-ROCmFP3 | 304B | 171.0 GB |
| DeepSeek-V4-Flash-Vision-Exp | 305B | 171.6 GB |
| keys-DeepSeekV4Flash-Vision-EXP-ablit | 305B | 171.6 GB |
| DeepSeek-V4-Flash | 291B | 163.7 GB |
| GLM-5.3-Flash | 321B | 180.6 GB |
| GPU | Model | Quant | Backend | Ctx | tok/s | prompt tok/s | VRAM GB | Source |
|---|---|---|---|---|---|---|---|---|
| CMP 170HX x2 · 64 GB | Hy3 | Q2_K_XL | llama.cpp 0.28.0 | 32768 | 27.6 ✓ | 250.7 | 115.9 | src |