X local-model bench digest — Sep 13, 2026
As of Sep 13, 2026 — 14 posts collected from X. 51 measured runs catalogued, 14 kept to the digest only. Latest source post Sep 13, 2026 11:20 UTC.
Every number below is what its author published, transcribed with the engine, quantization and context length they reported. Sources are linked on each entry. Only posts that state the full methodology (engine build, quantization, context length, hardware state, measurement method) reach the benchmark board; the rest stay here as leads.
RTX 5090 × Qwen3.8-27B
Sep 13, 2026 · by @witcheer
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 2954 | Q6_K | llama.cpp | — | 2940.6 tok/s |
| 11570 | Q6_K | llama.cpp | — | 3042.0 tok/s |
| — | Q6_K | llama.cpp | 109.0 tok/s | — |
| — | Q6_K | llama.cpp | 113.0 tok/s | — |
| — | Q6_K | llama.cpp | 139.0 tok/s | — |
| — | Q6_K | llama.cpp | 144.0 tok/s | — |
the loudest specific complaint in Alex Cheema's thread on why people don't run local models was prefill: a coding agent hands the model a long prompt and you wait for the first token, every request. two llama-server flags remove most of that wait, and both are free. I measured them on the model my always-on box serves every day. Qwen3.8-27B Q6_K, llama.cpp, one RTX 5090, batch 1, 304 requests, one variable at a…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, hardware state, measurement method).
Unspecified platform
Sep 13, 2026 · by @sudoingX
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 262K | — | — | 74.3 tok/s | 179.7 tok/s |
| 262K | — | — | 69.6 tok/s | 147.5 tok/s |
| 262K | Q4_0 | vulkan | 34.2 tok/s | 61.9 tok/s |
| 262K | — | rocm tensor split | 29.8 tok/s | 59.0 tok/s |
| 262K | — | — | 13.2 tok/s | 23.4 tok/s |
| 256K | Q6_K | — | 62.0 tok/s | 121.7 tok/s |
| 256K | Q4_K_M | — | 41.2 tok/s | 61.4 tok/s |
| 256K | — | nvlink | 35.7 tok/s | 59.1 tok/s |
| 256K | Q8_K_XL | — | 26.7 tok/s | 52.5 tok/s |
| 192K | — | — | 74.7 tok/s | 160.8 tok/s |
this is every card in the table with the mtp flag on and tok/s alone does not tell you if the card holds your context. here i grouped by the window it actually served at that speed, 73 rigs submitted in my open repo, five rows landed this morning. 262K rtx 5090 32gb, UD-Q4_K_XL: 74.3 tok/s → 179.7 tok/s (n4) rtx 5090 32gb, UD-Q5_K_XL: 69.6 tok/s → 147.5 tok/s (n6) radeon ai pro r9700 32gb, Q4_0, windows vulkan: 34.2…
Digest-only: hardware.name missing; model.name missing; incomplete methodology (engine+version, hardware state, measurement method).
DGX Spark × DeepSeek-V4.1-Flash-Next-DGX-Spark-512K
Sep 13, 2026 · by @sayyidfareed
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 524293 | — | — | 27.8 tok/s | — |
Great work by @0xBakeer on bringing DeepSeek-V4.1-Flash to a single DGX Spark. His runtime and optimization work gave me a strong foundation to build on. I’ve now taken that further with DeepSeek-V4.1-Flash-Next-DGX-Spark-512K: a self-contained K154 release built and qualified on one 128 GB GX10. My release packages a fixed, coding-ranked set of 154 experts per layer, removes the unused original routed-expert…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, hardware state, measurement method).
RTX 5070 Ti 16GB × Qwen3.8 27B
Sep 13, 2026 · by @rS_alonewolf
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 1852 | — | — | — | 1108.0 tok/s |
| 1852 | Q3_K_M | — | — | 1011.0 tok/s |
| 1852 | Q3_K_M | — | — | 1007.0 tok/s |
| 2108 | — | — | 60.3 tok/s | — |
| 2108 | Q3_K_M | — | 51.0 tok/s | — |
| 2108 | Q3_K_M | — | 46.0 tok/s | — |
■ 5070 Ti 16GBで Qwen3.8 27B の性能テスト 今度は速さ。引リツのベンチは何分かかったか？ ■ 1秒あたりの速さ 入力1,852tok・生成256tok、3本ともub640に揃えて3回反復。 プレフィル（読み込み tok/s） ・Qwen3.8 UD-Q3_K_XL：1,108 ・Qwen3.6 27B Q3_K_M：1,011 ・Qwen3.8 Q3_K_M：1,007 デコード（書き出し tok/s） ・Qwen3.8 UD-Q3_K_XL：60.3 ・Qwen3.6 27B Q3_K_M：51.0 ・Qwen3.8 Q3_K_M：46.0 プレフィルはほぼ横並びで、UDだけ1割速い。差が出るのはデコードで、UDはQ3_K_Mの1.3倍。 読みも書きもUDが速い。ここだけ見ればUD。 ■ 36課題を終えるまでの合計 ・GLM-5.3：2.4時間 ・Qwen3.6 27B：3.3時間 ・Qwen3.8…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, hardware state).
DGX Spark x2 × DeepSeek-V4.1-Flash REAP-256E
Sep 13, 2026 · by @Reederey
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | FP4 | — | 25.6 tok/s | — |
DeepSeek-V4.1-Flash REAP-256E running on 2× DGX Spark (GB10): 25.65 tok/s decode, 841,973-token KV pool. Fully reproducible. This is FP4, not a 3-bit or 2-bit EXL3 re-quant. What "REAP-256E" means is the routed-expert pool was pruned 384 → 256 per layer using Cerebras' REAP method — delete the experts that contribute least. Nothing was re-quantized. This is a beginning, not a finish line. More improvements planned…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, context length, hardware state, measurement method).
RX 7900 XTX × Kiwen1.1 27B
Sep 13, 2026 · by @xqliu
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | llama.cpp | 81.6 tok/s | — |
这两天看着 A 社的各种“抽象操作”，越来越怀疑云端大模型厂商的节操。 于是把本地方案认真测了一遍：RX 7900 XTX + llama.cpp + Kiwen1.1 27B + 自研 KV cache proxy。 实测长期生成速度 81.6 tok/s，MTP 接受率 66.1%，KV cache 命中率 95.9%，24 个代码/运维 Agent 任务 19/24 完成并通过自验证。 本地模型未必全面替代云端，但至少速度、成本、隐私和可控性，已经值得重新算一遍账了。
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
H200 x4 × DeepSeek-V4.1-Flash-UNCENSORED-FP8
Sep 13, 2026 · by @TeksEdge
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 1M | FP8 | sglang | 101.0 tok/s | — |
| 1M | FP8 | sglang | 113.0 tok/s | — |
| — | Q2_K | llama.cpp | 2.3 tok/s | — |
While we wait to see what Unsloth does with it, DeepSeek-V4.1-Flash has already been “uncensored”! The monster launched just 2 days ago. And the local inference race to mod it is already underway 👇 🧠 552B backbone 📚 +196B Engram memory ⚡ 8B active on input 🚀 16B active on output 📖 1M context 👁️ Vision 🏎️ DSpark speculative decoding 💾 ~510GB native checkpoint Dealignai modified the weights themselves and no custom…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, measurement method).
DGX Spark × Qwen3.8-27B
Sep 13, 2026 · by @Oluwaphilemon1
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | NVFP4 | — | 71.5 tok/s | — |
| — | NVFP4 | — | 57.0 tok/s | — |
A DGX Spark can apparently get 25% more performance without changing the DGX Spark. Qwen3.8-27B just hit 71.50 tok/s on a single GB10. The previous recipe was slower. Same model. Same hardware. Same NVFP4 target. So where did the extra performance come from? Mostly from changing the way inference happens. The new setup uses a calibrated NVFP4 speculative-decoding drafter, increased the draft window from 8 to 16…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, context length, hardware state, measurement method).
DGX Spark × Qwen3.8-27B
Sep 12, 2026 · by @TraffAlex · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 1M | — | ollama | 89.0 tok/s | — |
| 1M | — | ollama | 60.0 tok/s | — |
Did a deep-dive on where local AI actually stands right now. The last ~3 months of releases, benchmarks and hardware, in one post. No vibes — just numbers. ━━ THE MODELS (what people are actually running) Qwen3.8-27B — the star of the quarter. Scored 52 on the Artificial Analysis index, the same level as DeepSeek V4-Pro and GPT-5.6 Luna. First local model to hit frontier-class scores. Unsloth's dynamic quants fit it…
Digest-only: confidence 0.35 below 0.60; incomplete methodology (engine+version, quantization, hardware state, measurement method).
DGX Station GB300 × DeepSeek-V4.1-Flash
Sep 12, 2026 · by @JamesMeadlock
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 1M | — | — | 89.0 tok/s | — |
| 1M | — | — | 150.0 tok/s | — |
| 1M | — | — | 311.0 tok/s | — |
| 972K | — | — | — | 11435.3 tok/s |
DeepSeek-V4.1-Flash at 1M context on one DGX Station GB300 - 60GB experts in Grace Release: Sixty (2026-09-12) · Status: verified (2026-09-10/12) · v12: 89 tok/s single-stream prose · 140–160 tok/s on agent/code text · 311 agg tok/s at C16 (v11: 82 / 130–150 / 287) · 972K-token prompt prefilled in 85 s · Hermes tool-calling 10/10 https://t.co/bFiFpuCKii
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, hardware state, measurement method).
Unspecified platform × DeepSeek V4.1 Flash
Sep 12, 2026 · by @onusoz · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | — | 250.0 tok/s | — |
A threshold was crossed with DeepSeek V4.1 Flash, similar to when Claude Code launched, or the last Christmas of Agents. It's so fast and cheap, 200-250 tok/s on Novita Coupling this model with one of the bigger flagship models can get one much faster to the finishing line in any work So bullish for everyone to follow similar architectures
Digest-only: confidence 0.35 below 0.60; hardware.name missing; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
DGX Spark x8 × DeepSeek-V4.1-Flash
Sep 12, 2026 · by @light_foundry
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | MXFP4 expert / FP8 dense | sglang | 48.0 tok/s | — |
| — | MXFP4 expert / FP8 dense | sglang | 179.0 tok/s | — |
| 32768 | MXFP4 expert / FP8 dense | sglang | — | 4400.0 tok/s |
DeepSeek-V4.1-Flash on 8x DGX Spark, SGLang TP8/EP8. 48 tok/s single-stream prose 179 tok/s aggregate at c8 ~4.4K tok/s prefill at 32K 2.6M-token KV pool, 256K context Official MXFP4 expert / FP8 dense weights. Vision + tools on. Huge credit to @MiaAI_lab. Their 3x Spark recipe took my single-stream decode from 38 to 48 tok/s. Two additional flags unlocked the concurrency and prefill gains. Details below.
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, hardware state, measurement method).
Also in this window
The 12 most recent posts are shown in full above; 2 posts from the same window are listed here in short form, each with its source post.
- Cerebras system × Qwen3.8-27B — 1850.0 tok/s decode (2 measurements) · @analogalok
- DGX Spark x4 × DeepSeek-V4.1-Flash — 99.5 tok/s decode (9 measurements) · @danushman
How to read this page
This digest is an automated transcription of public posts. Runs with a stated engine build, quantization, context length, hardware state and measurement method are published straight to the benchmark board with a link to their source; the rest stay here. These are author-reported numbers, not an independent DiggTop re-measurement: every row carries its provenance so you can check the original, and a number that turns out to be misreported is corrected or removed. Our own testing follows the methodology. Spotted an error? Reply on the source post or submit a corrected run at /submit.
Sources
- @witcheer — Sep 13, 2026
- @sudoingX — Sep 13, 2026
- @sayyidfareed — Sep 13, 2026
- @rS_alonewolf — Sep 13, 2026
- @Reederey — Sep 13, 2026
- @xqliu — Sep 13, 2026
- @TeksEdge — Sep 13, 2026
- @Oluwaphilemon1 — Sep 13, 2026
- @TraffAlex — Sep 12, 2026
- @JamesMeadlock — Sep 12, 2026
- @onusoz — Sep 12, 2026
- @light_foundry — Sep 12, 2026
- @analogalok — Sep 12, 2026
- @danushman — Sep 12, 2026
FAQ
Are the numbers in this digest sourced?
They are transcribed from public X posts and the repositories those posts link to, with attribution and a source link on every entry. Rows that state the full methodology are published to the benchmark board automatically, so treat them as author-reported until a dispute is raised: the source link on every row is the check.
What does it take for a row to reach the sourced tier?
Engine and exact version, quantization, context length, hardware state and a described measurement method. Posts that state all five are published to the benchmark board as soon as the collector reads them, each row carrying its source link — nothing waits in a human review queue. Anything missing a condition stays in this digest.
Where does the data come from?
Public X posts about local-model inference measured on owned hardware, discovered by keyword search every two hours and linked back to the original thread. The digest is regenerated daily.