DiggTop
Menu
⌕

X local-model bench digest — Sep 12, 2026

As of Sep 12, 2026 — 25 posts collected from X. 81 measured runs catalogued, 4 published to the board, 21 kept to the digest only. Latest source post Sep 12, 2026 10:16 UTC.

Every number below is what its author published, transcribed with the engine, quantization and context length they reported. Sources are linked on each entry. A row that states the engine and version, the quantization, the context length and a specific GPU is labelled sourced on the benchmark board; anything missing one of those is partial, and partial rows never count in an average.

Unspecified platform × DeepSeek V4.1 Flash

Sep 12, 2026 · by @voluntas · third-party report

Context Quant Engine Decode Prefill
— — — 260.0 tok/s —

DeepSeek V4.1 Flash さん 260 tok/s でとる ... 。恐ろしい。

Digest-only: confidence 0.35 below 0.60; hardware.name missing; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).

DGX Spark × Qwen 3.8 Flash Next

Sep 12, 2026 · by @sudoingX

Context Quant Engine Decode Prefill
128K — — 44.4 tok/s —
51K int4 — 42.1 tok/s —
— — — 35.0 tok/s —
231K nvfp4 — 35.3 tok/s —

i spent two weeks living on every 100B+ model that fits one dgx spark 128gb and this is the list i wish existed the day the box landed. every option i could find, tested on this box, what survived is ranked below, all measured on my desk: 1. qwen 3.8 flash next, 180B with 11B active, 44.4 tok/s, holds its speed at depth (-35% at 63K), 1M context fits in memory (measured to 128K) @Alibaba_Qwen official…

Digest-only: no run with engine+version+quant+context+tps; incomplete methodology (engine+version, hardware state, measurement method).

DGX Spark × DeepSeek-V4.1-Flash

Sep 12, 2026 · by @0xBakeer

Context Quant Engine Decode Prefill
5K — — — 337.0 tok/s

Continuing on DeepSeek-V4.1-Flash on a single DGX Spark today. Published v0.4.0-wip a few minutes ago. 17-37 tok/s across nine workloads, prefill 337 tok/s on a 5k prompt, 44% of the routed experts resident in a 3-bit format, nothing read from SSD during decode (in this version). The last version had few bugs. The thing that actually fixed it wasn’t a kernel. The expert keep-set is a cache policy learned from a…

Digest-only: no run with engine+version+quant+context+tps; incomplete methodology (engine+version, hardware state, measurement method).

RTX 5090 × DeepSeek-V4.1

Sep 12, 2026 · by @TeksEdge

Context Quant Engine Decode Prefill
— — llama.cpp 5.1 tok/s —
— — llama.cpp 21.3 tok/s —

Well... THAT didn’t take long. I wrote that DeepSeek-V4.1 could already be converted to GGUF, but the hard part was still missing, which was essentially the runtime graph. Now a community llama.cpp fork has it actually generating tokens on a single RTX 5090 with 31.8 GiB VRAM + 125.7 GiB system RAM, while the rest of the ~500GB model lives on NVMe. The setup uses a real memory hierarchy, 👉 VRAM for hot experts → RAM…

Digest-only: no run with engine+version+quant+context+tps; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).

DGX Spark x2 × DeepSeek-V4.1-Flash

Sep 12, 2026 · by @seanphan

Context Quant Engine Decode Prefill
— FP4 triton 38.2 tok/s —
— FP4 triton 20.8 tok/s —

DeepSeek-V4.1-Flash on 2x DGX Spark hot-set + NVMe streaming (keep-31% FP4 resident). Tks @0xBakeer for the recipe - 38.2 tok/s decode c=2 - 20.8 tok/s single stream SSD is doing most of the work. Attention + Engram + the Triton FP4 MoE kernel are under 8% of decode time.

Digest-only: no run with engine+version+quant+context+tps; incomplete methodology (engine+version, context length, measurement method).

DGX Spark

Sep 12, 2026 · by @shi3z

Context Quant Engine Decode Prefill
— — — 47.6 tok/s —

Finally pulled it off! Hit 47.57 tok/s (decode) on a single DGX Spark with all experts loaded. Prefill takes a hit, sure, but for a solo box, not too shabby at all, right? https://t.co/yqareJRENw

Digest-only: model.name missing; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).

DGX Spark x4 × DeepSeek-V4.1-Flash

Sep 12, 2026 · by @yunwei37

Context Quant Engine Decode Prefill
— — vllm 50.1 tok/s —
— — vllm 144.7 tok/s —

deepseek 4.1 flash on 4 dgx spark ring without switch You don't need a additional switch for it 50.14 tok/s C1 mean (code 68.06); 144.71 tok/s at 8 request https://github.com/yunwei37/dgx-spark-4-ring-no-switch

Digest-only: no run with engine+version+quant+context+tps; incomplete methodology (engine+version, quantization).

DGX Spark x4 × DeepSeek V4.1 Flash

Sep 12, 2026 · by @Oluwaphilemon1

Context Quant Engine Decode Prefill
300K MXFP4 vllm 170.7 tok/s —
300K MXFP4 vllm 77.6 tok/s —
300K MXFP4 vllm 74.1 tok/s —

Four DGX Sparks running DeepSeek V4.1 Flash is starting to look less like a local AI experiment and more like a miniature inference cluster. The setup pushed 6 concurrent sessions to a combined 170.73 tokens per second on coding workloads. For perspective, that means 1,200 output tokens were generated in just 7.028 seconds in the recorded run. The interesting part is how the performance changes depending on the…

Digest-only: no run with engine+version+quant+context+tps; incomplete methodology (engine+version, hardware state).

A100 × DeepSeek v4.1 Flash

Sep 12, 2026 · by @drikin · third-party report

Context Quant Engine Decode Prefill
— FP4 — 33.0 tok/s —
— FP4 — 673.0 tok/s —

やばい、無邪気に天才を怒らせたらとんでもないことが起きてしまった！！ DeepSeek v4.1 Flashを動かしたくてFP4に対応していないA100を、33tok/sから673 tok/sまで持っていって気がつくと公式APIより速くなっていた話｜shi3z @shi3z https://note.com/shi3zblog/n/nd5fc5341b342?sub_rt=share_pw

Digest-only: confidence 0.35 below 0.60; incomplete methodology (engine+version, context length, hardware state, measurement method).

RTX PRO 6000 × Qwen3.8-Flash-Next

Sep 12, 2026 · by @wei_wang

Context Quant Engine Decode Prefill
— EXL3 mixed K4/K5, avg ~4.25 bit custom 160.7 tok/s —
— EXL3 mixed K4/K5, avg ~4.25 bit custom 268.1 tok/s —
— EXL3 mixed K4/K5, avg ~4.25 bit custom 418.6 tok/s —
— EXL3 mixed K4/K5, avg ~4.25 bit custom 645.3 tok/s —
— EXL3 mixed K4/K5, avg ~4.25 bit custom 975.4 tok/s —
— EXL3 mixed K4/K5, avg ~4.25 bit custom — 7600.0 tok/s

刚在我的RTX PRO 6000 96GB 上部署了 Qwen3.8-Flash-Next。 这是一个约 176B 总参数、每个 token 激活约 6B 的 MoE 模型，其中包括 125B 主模型和约 51B 的 PLE n-gram embedding table，同时保留完整 256K 上下文。 这么大的模型能跑进单张 96GB 显卡，靠的是分层处理。 MoE 专家权重使用 EXL3 混合 K4/K5 量化，平均约 4.25 bit。47.68GiB 的 PLE table 压成 FP8 后通过 mmap 放在 NVMe 上按需读取，Token embedding 放进系统内存，FP8 KV Cache 留给长上下文。 模型实际加载约占 71.99GiB 显存，整卡实测峰值约 91.4GiB，刚好控制在 96GB 以内。 Decode 和 prefill 速度超级猛，感觉在飞。 Prose c1 160.7 tok/s…

Digest-only: no run with engine+version+quant+context+tps; incomplete methodology (engine+version, context length, measurement method).

DGX Spark x2 × DeepSeek-V4.1-Flash

Sep 11, 2026 · by @sfxnz

Context Quant Engine Decode Prefill
— EXL3 2.0bpw-mcg vllm deepseekv41-flash-0909@sha256:d84a123255b822fc22… 23.0 tok/s —

Improvement to Deepseek v4.1-flash on 2x DGX sparks C=1 prose decode now at 23 tok/s up from 10 tok/s 🔥 Making this is a viable option to now run at TP=2 try it https://github.com/sfxnz/DeepSeek-V4.1-Flash-EXL3-vLLM-2x-DGX-Spark

Digest-only: no run with engine+version+quant+context+tps; incomplete methodology (context length).

GB300 x4 × DeepSeek V4.1 Flash

Sep 11, 2026 · by @sgl_project

Context Quant Engine Decode Prefill
— FP8 sglang 873.0 tok/s —

Day 0 support was only the beginning. ⚡ Within 24 hours of launch, we pushed performance to 873 tok/s at BS=1 on 4×GB300 through FP8 GEMM fast paths, kernel fusion and overlap, DSpark optimization, and MoE TP4. DeepSeek V4.1 Flash introduced major architectural changes, including a causal encoder-decoder design, CSA2 shared KV, Engram, mHC, and DSpark. SGLang supported them all from Day 0. Our new hands-on…

Digest-only: no run with engine+version+quant+context+tps; incomplete methodology (engine+version, context length, hardware state, measurement method).

Also in this window

The 12 most recent posts are shown in full above; 13 posts from the same window are listed here in short form, each with its source post.

How to read this page

This digest is a transcription of public posts. Rows that state the engine build, quantization, context length and hardware are published to the benchmark board with a link to their source; the rest stay here. These are author-reported numbers, not an independent DiggTop re-measurement. Our own testing follows the methodology. Own one of these measurements? Submit it through the three-step run at /submit.

Sources

FAQ

Are the numbers in this digest sourced?

They are transcribed from public X posts and the repositories those posts link to, with attribution and a source link on every entry. No: DiggTop does not re-run them. A row that states the full methodology is labelled sourced on the benchmark board; the rest are partial and count in nothing.

What does it take for a row to reach the sourced tier?

Engine and exact version, quantization, context length, a specific hardware configuration and a source link. Posts that state all of those are published to the board as soon as the collector reads them; anything missing one stays in this digest, marked partial.

Where does the data come from?

Public X posts about local-model inference measured on owned hardware, discovered by keyword search every two hours and linked back to the original thread. The digest is regenerated daily.

↑