Ornith-1.5-9B
8recorded runs
102.0best tok/s
5verified
What the Ornith-1.5-9B numbers mean
The fastest verified result on our board is 102.0 output tok/s on an RTX 5070 at Q4_K_M with a 2048-token context. That is about 9.8 ms per generated token — 500 tokens takes roughly 4.9 seconds. Prompt processing on the same record runs at 4046.7 tok/s, so a 2,000-token prompt is read in about 0.5 seconds. Across the 8 records we hold for this model the range is 22.9–102.0 tok/s on 8 GPUs, because quantisation, context length and the engine build all move the number.
How much VRAM does Ornith-1.5-9B need?
Ornith-1.5-9B has 10.0B parameters. These estimates use the same formula as the VRAM calculator: weights, plus about 15% for a moderate-context KV cache and runtime overhead, plus a small engine floor.
| Quantisation | Estimated VRAM | Smallest card that fits |
|---|---|---|
| Q4_K_M (4-bit) | 7.3 GB | 12 GB |
| Q8_0 (8-bit) | 13.0 GB | 16 GB |
| FP16 (16-bit) | 23.8 GB | 32 GB |
Fastest GPUs for this model (best tok/s)
- RTX 5070102.0
- RTX 308082.8
- RTX 306065.0
- CMP 170HX61.5
- RTX 2060 Super60.4
Models with a similar footprint
| Model | Params | Q4_K_M weights |
|---|---|---|
| GLM-4.6V-Flash | 10B | 5.6 GB |
| Qwen3.5-9B | 10B | 5.6 GB |
| Carnice-9b | 9B | 5.1 GB |
| DeepSeek-V4-Pro-Qwen3.5-9B-MTP | 9B | 5.1 GB |
| Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated-i1 | 9B | 5.1 GB |
| NVIDIA-Nemotron-Nano-9B-v2 | 9B | 5.1 GB |
| GPU | Model | Quant | Backend | Ctx | tok/s | prompt tok/s | VRAM GB | Source |
|---|---|---|---|---|---|---|---|---|
| RTX 5070 · 12 GB | Ornith-1.5-9B | Q4_K_M | llama.cpp b10449 @ 0d9ceae1 | 2048 | 102.0 ✓ | 4046.7 | — | src |
| RTX 3080 · 10 GB | Ornith-1.5-9B | Q6_K | llama.cpp b10472 | 131072 | 82.8 ✓ | 3397.4 | 8.5 | src |
| CMP 170HX · 64 GB | Ornith-1.5-9B | BF16 | vllm 0.27.1 | 131334 | 61.5 ✓ | 5305.1 | — | src |
| RTX 2060 Super · 8 GB | Ornith-1.5-9B | Q4_K_M | llama.cpp b10449 (0d9ceae1e38291035605613ab41a… | 8192 | 60.4 ✓ | 1360.1 | 5.4 | src |
| Intel Arc Pro B70 · 32 GB | Ornith-1.5-9B | Q8_0 | llama.cpp 9fee29e9435f; oneAPI 2026.1.1 | 8192 | 49.6 ✓ | — | — | src |
| RTX 3060 · 12 GB | Ornith-1.5-9B | Q4_K_M | llama.cpp — | 65536 | 65.0 … | 3114.4 | — | src |
| BC-250 · 8 GB | Ornith-1.5-9B | Q4_K_M | llama.cpp b10705-vulkan | 12554 | 42.1 … | 242.3 | — | src |
| RTX 2080 · 8 GB | Ornith-1.5-9B | Q4_K_M | llama.cpp — | 131072 | 22.9 … | 292.6 | 6.8 | src |