DiggTop
Menu
⌕

X local-model bench digest — Oct 7, 2026

As of Oct 7, 2026 — 35 posts collected from X. 83 measured runs catalogued, 1 published to the board, 34 kept to the digest only. Latest source post Oct 7, 2026 13:03 UTC.

Every number below is what its author published, transcribed with the engine, quantization and context length they reported. Sources are linked on each entry. Only posts that state the full methodology (engine build, quantization, context length, hardware state, measurement method) reach the benchmark board; the rest stay here as leads.

RTX 5060 Ti × Bonsai 2

Oct 7, 2026 · by @sudoingX

Context Quant Engine Decode Prefill
— — — 124.7 tok/s —
60K — — 42.0 tok/s —
60K — — 10.5 tok/s —

4 agents running concurrently on a single 5060 ti, watch them build 4 games anon. what one 16gb card pushes: 124.7 tok/s, 4 short chats running at once 42 tok/s, 4 agents each holding ~60k of context 10 to 11 tok/s per agent at that depth bonsai 2 is qwen 3.8 27b squeezed to 5.95gb, so here's what it does on the cards might own: 12gb like a 3060, one agent at about 50 tok/s 16gb like a 5060 ti, one agent at about 67…

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, measurement method).

RTX 3090 × qwen

Oct 7, 2026 · by @starmexxx · third-party report

Context Quant Engine Decode Prefill
229K — mlx 1035.0 tok/s —

when i cancelled claude max, anthropic's form asked me why i was leaving. none of the boxes said "my iphone does it now" i was paying dario amodei $200 and sam altman another $200 every month. then i found 5 open source repos on github and gave every device i already owned a new job THE MAC ON MY DESK was: a browser tab pointed at claude now: claude code running on its own model, on my desk → mlx-serve (378 stars)…

Digest-only: confidence 0.35 below 0.60; incomplete methodology (engine+version, quantization, hardware state, measurement method).

DGX Spark x2 × Qwen3.8 Flash Next

Oct 7, 2026 · by @NFT_Chen

Context Quant Engine Decode Prefill
— NVFP4 tensorfold 69.0 tok/s —
— NVFP4 tensorfold 211.0 tok/s —
— INT4-AutoRound tensorfold 88.0 tok/s —
— INT4-AutoRound tensorfold 279.0 tok/s —

🔥厉害了！本地满血版、提速版 Qwen3.8 Flash Next 任你选！两台 DGX Spark 就能跑，还带 1M 上下文！ TensorFold 直接给两条路，按需求选： 1️⃣NVFP4 满血版
🔹官方量化，每 token 10 专家、6B 激活，该有的能力都在
🔹单流约 69 tok/s，8 路并发合计约 211 tok/s
🔹适合写代码、看长文档、要质量优先的人
权重：https://t.co/6lnxTYBrEN 2️⃣INT4-AutoRound 提速版
🔹专家减到 5 个、激活 4.8B，INT4 更轻，桌面级机器也能顶住
🔹单流约 88 tok/s，8 路并发合计约 279 tok/s，明显更快
🔹综合能力大约掉 10%，工具调用几乎不掉，日常够用
权重：https://t.co/V7jCr1lRL5 质量选满血，速度选 INT4。两台 Spark，本地就能当主力模型用。 #Qwen #DGXSpark…

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, context length, hardware state, measurement method).

DGX Spark × Qwen3.8-Flash-Next

Oct 7, 2026 · by @sudoingX

Context Quant Engine Decode Prefill
— 4 bit QAD tensorfold 65.0 tok/s —
— FP8 vllm 58.0 tok/s —
— FP8 vllm 24.0 tok/s —

i asked how tensorfold holds up, then i ran it all night myself, quality held anon, this engine is legit and one dgx spark kept up with my two in speed humaneval+ 95.1% vs 95.7% on my vllm serve mbpp+ 79.9% vs 81.0% ifeval 86.0% vs 86.5% all within noise, 1,083 problems paired one by one drafted output byte-identical to serial, 70 of 70 code at 65 tok/s on 1 spark vs 58 tok/s on my 2 tensorfold ran a 4 bit qad build…

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, context length, hardware state, measurement method).

M5 Max 128GB × GLM-5.3-Flash

Oct 7, 2026 · by @Beamsters1

Context Quant Engine Decode Prefill
1M — sushi 1.2.0 — 850.0 tok/s
1M — sushi 1.2.0 45.0 tok/s —
— 2.5bpw sushi 1.2.0-dev 44.7 tok/s —
— 2.5bpw sushi 1.2.0-dev2 55.4 tok/s —
— 2.4bpw sushi 1.2.0 54.6 tok/s —

Sushi 🍣 1.2.0 is here with GLM 5.3 Flash that fits your 128GB Mac. 🍣 Final KLD ~0.074 and 89GiB 🍣 Tool-eval-bench 92/100 - ⭐️⭐️⭐️⭐️⭐️ 🍣 Prefill 850 tok/s, decode 4050 tok/s (M5 Max with DFlash2) 🍣 1m context ⭐️ https://github.com/beamivalice/sushi https://t.co/rSd8CEzFc3

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (context length).

RTX 3090 × GLM-5.3-Flash

Oct 7, 2026 · by @0xSero

Context Quant Engine Decode Prefill
— exl3 — 35.0 tok/s —
— exl3 — — 1136.0 tok/s
— exl3 — 76.5 tok/s —

GLM-5.3-Flash exl3 on CPU is going way faster, now thanks to my friend's amazing 4 months of crunching on this. 1x rtx 3090 + 50GB DDR4 + 100GB NVMe - decode 26 tok/s -> 35 tok/s - prefill = 1018 -> 1136 tok/s - concurrency 4 = 76.5 tok/s Intelligence for all. Freedom https://t.co/Y1LXhabjHa

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, context length, hardware state, measurement method).

RTX 3090 × Qwen3.8-Flash-Next

Oct 7, 2026 · by @Oluwaphilemon1 · third-party report

Context Quant Engine Decode Prefill
— IQ3_S — 140.0 tok/s —

If you’re running Qwen3.8-Flash-Next on an RTX 3090 with 64GB of RAM, IQ3_S might be worth using instead of IQ3_XXS. Normally, moving from IQ3_XXS to IQ3_S means accepting a larger model footprint and lower inference speed in exchange for better quality. But Strata changes the equation. I switched from: Qwen3.8-Flash-Next-GSQ-RSO-IQ3_XXS to: Qwen3.8-Flash-Next-GSQ-RSO-IQ3_S The model is bigger. The quality is…

Digest-only: confidence 0.35 below 0.60; incomplete methodology (engine+version, context length, hardware state, measurement method).

Unspecified platform × Qwen3.8-Flash-Next

Oct 7, 2026 · by @Oluwaphilemon1 · third-party report

Context Quant Engine Decode Prefill
— — llama.cpp 25.0 tok/s —
— — llama.cpp 50.0 tok/s —
— — llama.cpp 40.0 tok/s —
— — llama.cpp 100.0 tok/s —
— — llama.cpp 80.0 tok/s —

Qwen3.8-Flash-Next was doing around 25 tok/s. Now it can push past 50 tok/s. Qwen3.6-35B Q8 went from roughly 40 tok/s to 100 tok/s. Qwen3.8-27B went from around 40 tok/s to 80 tok/s. The models didn’t suddenly get smarter. The GPUs didn’t change. The inference stack did. A recent cafe-llama.cpp update is extracting much more performance from Qwen’s Multi-Token Prediction setup, with reported decoding gains…

Digest-only: confidence 0.35 below 0.60; hardware.name missing; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).

DGX Spark x4 × GLM-5.3

Oct 7, 2026 · by @majewskizby

Context Quant Engine Decode Prefill
256K FP4 KV + NVFP4 weights for atten sparkdash 35.0 tok/s —
256K FP4 KV + NVFP4 weights for atten sparkdash 42.0 tok/s —
256K FP4 KV + NVFP4 weights for atten rigmark 28.4 tok/s —
256K FP4 KV + NVFP4 weights for atten rigmark 39.0 tok/s —
256K FP4 KV + NVFP4 weights for atten rigmark 44.4 tok/s —

Full GLM-5.3 (753B) on 4x DGX Spark, small update 🐘 • Context 64K → 256K • Sparkdash: prose c1 32.7 → 35.0 tok/s, code 37.9 → 42.0 • RigMark c1 (thinking on): prose 25.4 → 28.4, code 36.4 → 39.0, structured 41.3 → 44.4 FP4 KV + NVFP4 weights for attention/shared/MTP, and memory floors we finally measured instead of guessed. Thanks to @MiaAI_lab , @tonyd2wild, kindling and ajclark. This release builds on your work…

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, measurement method).

Unspecified platform × Qwen3.8-FLASH-NEXT

Oct 7, 2026 · by @uzuki425

Context Quant Engine Decode Prefill
6800 — — — 1657.0 tok/s
6800 — — — 855.0 tok/s

Qwen3.8-FLASH-NEXTのPrompt-evalが劇的に向上。 約855〜1657 tok/s(6.8k tok: 30.5秒 → 4.1秒、約7倍) PRO-6000を大幅に上回りました。もう切り変えるしかないですね。

Digest-only: hardware.name missing; incomplete methodology (engine+version, quantization, hardware state, measurement method).

DGX Spark x2 × DeepSeek-V4.1-Flash

Oct 7, 2026 · by @bertholomusai

Context Quant Engine Decode Prefill
1M EXL3 2.9bpw vllm 100.0 tok/s —
1M EXL3 2.9bpw vllm 140.0 tok/s —
1M EXL3 2.9bpw vllm 62.0 tok/s —
1M EXL3 2.9bpw vllm 125.0 tok/s —
8K EXL3 2.9bpw vllm — 2000.0 tok/s
128K EXL3 2.9bpw vllm — 2190.0 tok/s
128K EXL3 2.9bpw vllm 128.7 tok/s —

MORE SPEED! v0.5 DeepSeek-V4.1-Flash on 2× DGX Spark Tensorfold TP2 Weights: @MiaAI_lab EXL3 2.9bpw of deepseek-ai/DeepSeek-V4.1-Flash (MIT). • full 1M-token window, 4 parallel slots • 1 stream: ~100 tok/s code, 140 structured, 62 prose • 4 streams sustained: 125 tok/s • prefill 2,000–2,190 tok/s at 8K–128K • decode after a 128K prompt: 128.7 tok/s

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, hardware state, measurement method).

Unspecified platform

Oct 7, 2026 · by @nmatt0 · third-party report

Context Quant Engine Decode Prefill
1M — — 55.0 tok/s —

Just gave GLM 5.3 Flash USB access to a cell module on this gas tank monitor and let it go crazy 😍 All Local on my 2 Sparks. No more giving my alpha to frontier labs. I'm getting around 55 tok/s with 1M context window. @MiaAI_lab recipe FTW. GLM 5.3 Flash with its vision is also great at understanding pictures of PCBs and knowing what components are in view. Even can point you towards likely debug interfaces! Its a…

Digest-only: confidence 0.35 below 0.60; hardware.name missing; model.name missing; incomplete methodology (engine+version, quantization, hardware state, measurement method).

Also in this window

The 12 most recent posts are shown in full above; 23 posts from the same window are listed here in short form, each with its source post.

How to read this page

This digest is an automated transcription of public posts. Runs with a stated engine build, quantization, context length, hardware state and measurement method are published straight to the benchmark board with a link to their source; the rest stay here. These are author-reported numbers, not an independent DiggTop re-measurement: every row carries its provenance so you can check the original, and a number that turns out to be misreported is corrected or removed. Our own testing follows the methodology. Spotted an error? Reply on the source post or submit a corrected run at /submit.

Sources

FAQ

Are the numbers in this digest sourced?

They are transcribed from public X posts and the repositories those posts link to, with attribution and a source link on every entry. Rows that state the full methodology are published to the benchmark board automatically, so treat them as author-reported until a dispute is raised: the source link on every row is the check.

What does it take for a row to reach the sourced tier?

Engine and exact version, quantization, context length, hardware state and a described measurement method. Posts that state all five are published to the benchmark board as soon as the collector reads them, each row carrying its source link — nothing waits in a human review queue. Anything missing a condition stays in this digest.

Where does the data come from?

Public X posts about local-model inference measured on owned hardware, discovered by keyword search every two hours and linked back to the original thread. The digest is regenerated daily.

↑