X local-model bench digest — Oct 14, 2026
As of Oct 14, 2026 — 34 posts collected from X. 111 measured runs catalogued, 1 published to the board, 33 kept to the digest only. Latest source post Oct 14, 2026 10:39 UTC.
Every number below is what its author published, transcribed with the engine, quantization and context length they reported. Sources are linked on each entry. Only posts that state the full methodology (engine build, quantization, context length, hardware state, measurement method) reach the benchmark board; the rest stay here as leads.
RTX 5090 × BlackwellBoy Abliterated MiniCPM-V 4.7 35B-A3B
Oct 14, 2026 · by @Blackwellboy
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 65536 | EXL3 4.0 body-bpw | exl3 | 123.4 tok/s | — |
| 131072 | EXL3 4.0 body-bpw | exl3 | 110.3 tok/s | — |
| 256K | EXL3 4.0 body-bpw | exl3 | 97.6 tok/s | — |
35B. IMAGE + VIDEO + TEXT. ONE RTX 5090. And ~98 tokens/sec decoding after a 261,111-token prompt. 👀 Meet BlackwellBoy Abliterated MiniCPM-V 4.7 35B-A3B, converted to EXL3 with a reusable multimodal runtime adapter. Heres what the current V1 actually delivers: ⚡ LONG CONTEXT, MEASURED Median decode speed across 3 runs per context: • 64K: 123.37 tok/s • 128K: 110.28 tok/s • Near 256K: 97.57 tok/s At the largest…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version).
DGX Spark × Qwen3.8-Flash-Next
Oct 14, 2026 · by @ViC305
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | EXL3 3.05 bpw | exllamav2 | 88.6 tok/s | — |
| — | EXL3 3.05 bpw | exllamav2 | 55.9 tok/s | — |
| — | — | exllamav2 | 69.8 tok/s | — |
| — | EXL3 3.05 bpw | exllamav2 | 35.3 tok/s | — |
| — | — | exllamav2 | 40.0 tok/s | — |
| 7200 | EXL3 3.05 bpw | exllamav2 | 26.2 tok/s | — |
| 7200 | — | exllamav2 | 25.8 tok/s | — |
🚀 𝗧𝗪𝗢 𝗠𝗢𝗗𝗘𝗟𝗦 𝗢𝗡𝗘 𝗦𝗣𝗔𝗥𝗞! One main frontier level agent running Qwen3.8-Flash Next. Up to eight concurrent sub agents running MiniCPM-V 4.7 35B-A3B. A shared 1M-token worker KV pool. All on ONE NVIDIA DGX Spark. 🔥 I’m launching a new recipe series built around a different question: Not just “What’s the biggest model I can fit?” But “How much useful work can this machine actually do?” Not two unrelated chat windows…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version).
Unspecified platform
Oct 14, 2026 · by @_nodelay · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | — | 11.0 tok/s | — |
| — | — | — | 60.0 tok/s | — |
저도 망설임 없이 gpt pro 를 포기한 이유가 고성능의 토큰을 뿌려주는 로컬 머신 덕분이죠. 이걸 살 때는 구독이 그래도 더 싼거 아닌가 싶었지만 gpt sol 이 11 tok/s 나오는 걸 보고 한심해졌습니다. 로컬에서 돌아가는 GLM 5.3 Flash 는 60 tok/s 넘게 나온다구!!
Digest-only: confidence 0.35 below 0.60; hardware.name missing; model.name missing; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
GB300 × DeepSeek V4.1 Flash
Oct 14, 2026 · by @ArtificialAnlys
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | aa-agentperf | 670.0 tok/s | — |
We’ve added DeepSeek V4.1 Flash to AA-AgentPerf, with results for NVIDIA and AMD hardware. At 100 tokens/s per user, NVIDIA GB300 is in first place with throughput up to 261M tokens/s per MW. AA-AgentPerf is our agentic inference performance benchmark. It replays real agentic coding sessions, including prompt caching and tool-call delays, to give a realistic measure of inference performance in day-to-day agentic…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, context length, hardware state).
Mac Studio (M4 Max) × Qwen3.6-35B-A3B
Oct 14, 2026 · by @hokazuya
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 1K | Q4_K_XL | llama.cpp 781dbc5 | 226.9 tok/s | — |
| 4K | Q4_K_XL | llama.cpp 781dbc5 | 220.2 tok/s | — |
| 400 | Q4_K_XL | llama.cpp 781dbc5 | 207.6 tok/s | — |
ご要望にお応えして？ Qwen3.6-35B-A3Bに対応しました。 https://github.com/hokar3361/moe-on-a-desk MacStudioで、1K~4Kの短いコンテキスト、コード出力で、200 tok/s位が実測で得られています。 llama .cppのパッチなので、まずllama. cppで試した後にやるといいかも。
Full methodology stated (engine, quantization, context, hardware state, method) — 3 added to the benchmark board.
PlayStation 5 × Qwen 3.8 27B
Oct 14, 2026 · by @chasmmmmmmmmmmm
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | — | 20.0 tok/s | — |
NOT EVEN THE PLAYSTATION 5 IS SAFE 🚨 Someone jailbroke a PS5 and got Qwen 3.8 27B running on its GPU at 20 tok/s. That's a real 27B AI model running locally on a games console. The same model on the PS5's CPU did about one token a minute. Meanwhile, DDR4 prices are up 24% in 3 months, AMD just raised GPU memory prices, and 5090s are basically gone. PS5 coming to a datacenter near you.
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
Unspecified platform × Qwen
Oct 14, 2026 · by @GuideboardLabs · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 128K | Q4_K_M | — | 25.4 tok/s | — |
Qwen 3.6 35B-A3B has been dethroned A model that runs 30 of its 40 layers as convolutions instead of attention just took the top spot on the Agon bench. @liquidai LFM2.5-24B-A2B is their own architecture, and it is not another Qwen finetune. 24B total with 2B active per token, 64 experts routed top-4, and its own 128K tokenizer, expanded in place from LFM2's 65K. The Qwen models on this board use 151,936 tokens on…
Digest-only: confidence 0.35 below 0.60; hardware.name missing; incomplete methodology (engine+version, hardware state, measurement method).
RTX 3090 × Qwen 3.8 27B
Oct 14, 2026 · by @sudoingX
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | — | 31.0 tok/s | — |
| — | — | — | 41.3 tok/s | — |
| — | Q2_K | — | 52.4 tok/s | — |
| — | Q2_K | — | 85.6 tok/s | — |
| — | — | — | 34.0 tok/s | — |
| — | — | — | 84.4 tok/s | — |
| — | — | — | 49.1 tok/s | — |
| — | — | — | 81.1 tok/s | — |
| — | — | — | 55.6 tok/s | — |
| — | — | — | 92.9 tok/s | — |
if you own an rtx 3090 or you're thinking of buying one, check this repo first, it will save you a few hours of head scratching this is the qwen 3.8 27b dense mtp repo i released two months ago, and since then it's been filled with community speed rows, so much valuable data for single gpu, multi gpu and weird setups here's what the 3090 rows alone tell you, mtp off to mtp on stock 3090, 31.0 tok/s → 41.3 tok/s oc…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, hardware state, measurement method).
DGX Spark x4 × GLM-5.3-Flash
Oct 14, 2026 · by @krunkosaurus
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 6K | 4-bit EXL3 | tensorfold TP4 | 128.0 tok/s | — |
| 1M | 4-bit EXL3 | tensorfold TP4 | 110.0 tok/s | — |
| — | 4-bit EXL3 | tensorfold TP4 | 249.0 tok/s | — |
GLM-5.3-Flash on 4 DGX Sparks in a ring. No switch, just direct 200G cables. - 128 tok/s decode at 6K context - 110 tok/s at 1M context - 249 tok/s across 8 requests at once Thinking max, 4-bit EXL3, TensorFold TP4. Huge thanks to @MiaAI_lab for the recipe. More details in https://t.co/Y2TqPS7q77
Digest-only: incomplete methodology (measurement method).
DGX Spark × RED-SNOW-5.3-FLASH
Oct 14, 2026 · by @yume_arasaki
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 262K | 2.49bpw | — | 45.2 tok/s | — |
| 262K | 2.49bpw | — | 45.7 tok/s | — |
| 262K | 2.49bpw | — | 35.9 tok/s | — |
| 262K | 2.49bpw | — | 27.5 tok/s | — |
| 33K | 2.49bpw | — | 32.0 tok/s | — |
| 8K | 2.49bpw | — | 50.1 tok/s | — |
| 240K | 2.49bpw | — | 44.4 tok/s | — |
| 8K | 2.49bpw | — | — | 450.0 tok/s |
| 240K | 2.49bpw | — | — | 640.0 tok/s |
| 262K | 2.49bpw | — | 44.0 tok/s | — |
| 262K | 2.49bpw | — | 17.1 tok/s | — |
| 262K | 2.49bpw | — | 22.9 tok/s | — |
Part 2. The speed run. What RED-SNOW on one DGX Spark actually delivers, workload by workload, and the one cost that matters at depth. Recipe by @ViC305 , One Spark, 2.49bpw, 97.4 GiB, 262k context. GLM 5.3 fully uncensored cybersecurity capabilities. Recap of part 1: RED-SNOW-5.3-FLASH is an abliterated build of GLM-5.3-Flash, 321B parameters, by Blackfrost AI, quantized by Cruz. The abliteration is in the weights…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version).
Unspecified platform × Bonsai
Oct 14, 2026 · by @CaryPalmerr
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 240K | — | — | 15.0 tok/s | — |
Bonsai 8GB GPU breakthroughs landing from benchmark results! Now at 15tok/s at the 240k context threshold! Insane! A week ago this was at 3 tok/s at 120k. I genuinely think it's worth the full context window after this all checks out and lands. Will report back in some hours. https://t.co/Qh2LP54aMZ
Digest-only: hardware.name missing; incomplete methodology (engine+version, quantization, hardware state, measurement method).
DGX Spark x2 × DeepSeek-V4.1-Flash
Oct 14, 2026 · by @wei_wang
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 262K | — | custom | 103.0 tok/s | — |
| 262K | — | custom | 58.3 tok/s | — |
| 262K | — | custom | 141.3 tok/s | — |
| 262K | — | custom | 44.5 tok/s | — |
| 262K | — | tensorfold | 100.5 tok/s | — |
| 32768 | — | custom | — | 1773.0 tok/s |
| 32768 | — | tensorfold | — | 2276.0 tok/s |
DeepSeek V4.1 Flash 现在两台 DGX Spark 也能跑了，配方和权重今天都公开了。 官方权重 510 GB，两台 Spark 加起来只有 256GB 内存。这份配方是这么塞进去的： 1. 15,360 个路由专家按敏感度分别量化到 2 到 5 bit，平均 2.70 bit，从 289 GB 压到约 184 GB 2. 203 GB 的 Engram 表不进内存，查的时候直接从 NVMe 读，所以每台要留约 430 GB 空闲硬盘 3. 两台走 ConnectX-7 线直连，TP=2 单路 decode，开投机解码：写代码 103 tok/s，散文 58.3，结构化输出 141.3。关掉投机解码在 44.5 左右。和原版比，top-1 一致率 95.57%。为了 262K 上下文，默认只开 2 路并发。 作者在同样两台机器上也测了 TensorFold 跑 Mia 的 2.9 bit 版本，写代码…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, measurement method).
Also in this window
The 12 most recent posts are shown in full above; 22 posts from the same window are listed here in short form, each with its source post.
- DGX Spark x2 × DeepSeek-V4.1-Flash — 141.0 tok/s decode (2 measurements) · @sudoingX
- RTX 3090 × GLM 5.3 Flash — 10.0 tok/s decode (1 measurement) · @itsmetamike
- M1 MacBook × Qwen3.8-27B — 5.5 tok/s decode (3 measurements) · @Oluwaphilemon1
- RTX 5090 × Qwen3.8-Flash-Next — 204.0 tok/s decode (2 measurements) · @niklaslenz_ai
- Unspecified platform × Liquid AI 24B-A2B — 47.8 tok/s decode (2 measurements) · @qxoticai
- Unspecified platform × LFM2.5-24B-A2B — 184.0 tok/s decode (4 measurements) · @MikelEcheve
- M4 Mac Mini x1 + RTX 3090 x1 + Chestnut igpu × Qwen3.8-next-flash — 25.0 tok/s decode (2 measurements) · @0xSero
- Arc Pro B70 × Qwen3.8-27B — 166.0 tok/s decode (4 measurements) · @dumblebackdoor
- DGX Spark × MiniCPM-V 4.7 35B-A3B — 125.0 tok/s decode (3 measurements) · @ViC305
- DGX Spark × deepseek v4.1 on — 141.0 tok/s decode (2 measurements) · @sudoingX
- M3 Ultra × LFM2.5-24B-A2B — 205.0 tok/s decode (4 measurements) · @Nativ_AI
- RTX 5090 × Mini CPM-V 4.7 — 214.0 tok/s decode (1 measurement) · @KyleHessling1
- RTX 3060 × LFM2.5-24B-A2B — 87.0 tok/s decode (2 measurements) · @DogukanUrker
- Unspecified platform × DeepSeek 4.1 flash — 1000.0 tok/s decode (2 measurements) · @mmastrac
- Unspecified platform × qwen — 17.4 tok/s decode (3 measurements) · @sudoingX
- RTX 3070 × MiniCPM-V 4.7 35B-A3B — 24.3 tok/s decode (4 measurements) · @bountyAIhunter
- Unspecified platform × Qwen3.8-Flash-Next — 1579.0 tok/s prefill (3 measurements) · @filicroval
- DGX Spark (GB10) x4 × GLM-5.3 — 64.0 tok/s decode (6 measurements) · @CK2084
- iPhone 17 Pro × Gemma 4 e2b — 56.5 tok/s decode (3 measurements) · @sudoingX
- Lucebox x2 × DeepSeek V4.1 Flash — 103.0 tok/s decode (4 measurements) · @luceboxai
- V100 32GB x2 × GLM-5.3-Flash — 30.5 tok/s decode (2 measurements) · @UtaAoya
- MacBook M5 24GB × MiniCPM-V-4.7-35B-A3B — 40.0 tok/s decode (1 measurement) · @mfranz_on
How to read this page
This digest is an automated transcription of public posts. Runs with a stated engine build, quantization, context length, hardware state and measurement method are published straight to the benchmark board with a link to their source; the rest stay here. These are author-reported numbers, not an independent DiggTop re-measurement: every row carries its provenance so you can check the original, and a number that turns out to be misreported is corrected or removed. Our own testing follows the methodology. Spotted an error? Reply on the source post or submit a corrected run at /submit.
Sources
- @Blackwellboy — Oct 14, 2026
- @ViC305 — Oct 14, 2026
- @_nodelay — Oct 14, 2026
- @ArtificialAnlys — Oct 14, 2026
- @hokazuya — Oct 14, 2026
- @chasmmmmmmmmmmm — Oct 14, 2026
- @GuideboardLabs — Oct 14, 2026
- @sudoingX — Oct 14, 2026
- @krunkosaurus — Oct 14, 2026
- @yume_arasaki — Oct 14, 2026
- @CaryPalmerr — Oct 14, 2026
- @wei_wang — Oct 14, 2026
- @sudoingX — Oct 14, 2026
- @itsmetamike — Oct 13, 2026
- @Oluwaphilemon1 — Oct 13, 2026
- @niklaslenz_ai — Oct 13, 2026
- @qxoticai — Oct 13, 2026
- @MikelEcheve — Oct 13, 2026
- @0xSero — Oct 13, 2026
- @dumblebackdoor — Oct 13, 2026
- @ViC305 — Oct 13, 2026
- @sudoingX — Oct 13, 2026
- @Nativ_AI — Oct 13, 2026
- @KyleHessling1 — Oct 13, 2026
- @DogukanUrker — Oct 13, 2026
- @mmastrac — Oct 13, 2026
- @sudoingX — Oct 13, 2026
- @bountyAIhunter — Oct 13, 2026
- @filicroval — Oct 13, 2026
- @CK2084 — Oct 13, 2026
- @sudoingX — Oct 13, 2026
- @luceboxai — Oct 13, 2026
- @UtaAoya — Oct 13, 2026
- @mfranz_on — Oct 13, 2026
FAQ
Are the numbers in this digest sourced?
They are transcribed from public X posts and the repositories those posts link to, with attribution and a source link on every entry. Rows that state the full methodology are published to the benchmark board automatically, so treat them as author-reported until a dispute is raised: the source link on every row is the check.
What does it take for a row to reach the sourced tier?
Engine and exact version, quantization, context length, hardware state and a described measurement method. Posts that state all five are published to the benchmark board as soon as the collector reads them, each row carrying its source link — nothing waits in a human review queue. Anything missing a condition stays in this digest.
Where does the data come from?
Public X posts about local-model inference measured on owned hardware, discovered by keyword search every two hours and linked back to the original thread. The digest is regenerated daily.