DiggTop
Menu
⌕

X local-model bench digest — Sep 15, 2026

As of Sep 15, 2026 — 23 posts collected from X. 78 measured runs catalogued, 2 published to the board, 21 kept to the digest only. Latest source post Sep 15, 2026 08:00 UTC.

Every number below is what its author published, transcribed with the engine, quantization and context length they reported. Sources are linked on each entry. Only posts that state the full methodology (engine build, quantization, context length, hardware state, measurement method) reach the benchmark board; the rest stay here as leads.

Unspecified platform × DeepSeek V4.1 Flash

Sep 15, 2026 · by @xueyu1125 · third-party report

Context Quant Engine Decode Prefill
— — — 1850.0 tok/s —

当大家还在为DeepSeek V4.1 Flash 和 Gemini 3.8 Flash 300+ tok/s速度欢呼时，Cerebras 已经把Qwen3.8-27B 拉到 1850 tok/s了 ，太离谱了😱 正常GPU每生成一个token，需要反复从HBM读取模型权重，Qwen3.8-27B如果要达到 1850 tok/s, HBM带宽就需要达到 27GB ✖️ 1850 ≈ 50TB/s（batch=1、8-bit 权重），HBM根本达不到这么高带宽 Cerebras 把整片晶圆做成一颗 WSE，有 44 GB SRAM，权重都在 SRAM 中，不走HBM就不会被HBM带宽限制，直接绕过了权重搬运的限制，才达到1850 tok/s速度 Cerebras官方的文档我放下面👇

Digest-only: confidence 0.35 below 0.60; hardware.name missing; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).

Unspecified platform × DeepSeek v4.1 Flash

Sep 15, 2026 · by @MiaAI_lab

Context Quant Engine Decode Prefill
— — — 315.0 tok/s —
— — — — 52000.0 tok/s

DeepSeek v4.1 Flash through API costs less than $3 for over 500 Million tokens. Even more crazy: I measured a peak of ~315 tok/s decode and ~52k prefill https://t.co/TuKnmlNtGr

Digest-only: hardware.name missing; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).

RTX 4090 × Qwen3.8-27B

Sep 15, 2026 · by @Oluwaphilemon1

Context Quant Engine Decode Prefill
65536 — vllm pinned version with 13 patches 140.4 tok/s —
65536 — vllm pinned version with 13 patches 136.8 tok/s —
38K — vllm pinned version with 13 patches 141.2 tok/s —
38K — vllm pinned version with 13 patches 133.9 tok/s —

The RTX 4090 is almost four years old. And someone just pushed Qwen3.8-27B to 140.4 tok/s on one. That is a pretty good reminder that local AI performance isn’t determined by the GPU alone. The hardware in this test is the RTX 4090, launched in October 2022 for $1,599. The model is Qwen3.8-27B. The real secret is the inference stack. The recipe came from a community build originally optimized for the RTX 3090. That…

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (hardware state).

RTX 3090 × Qwen3.8-27B

Sep 15, 2026 · by @yibie

Context Quant Engine Decode Prefill
245760 IQ4_XS-M llama.cpp llamAmpere v0.3 99.0 tok/s —
100K IQ4_XS-M llama.cpp llamAmpere v0.3 93.0 tok/s —

llama.cpp 的 Ampere 分支作者 JakeATX 让一张 3090 跑出接近三位数的 tok/s，上下文开到两百多 K，比 stock llama.cpp 快近五成。 速度来自上游没有的 MTP 投机解码，和为这张卡手写的 SM86 融合 kernel。 llamAmpere v0.3 是他给 Ampere 世代（尤其 3090 / 3090 Ti）做的 llama.cpp 分支，跑 Qwen3.8-27B： 99 tok/s（干净的 agent 与编码 fixture，temperature 1），是 stock llama.cpp 的 1.46 倍、上一版 v0.2 的 1.28 倍；KV 深度 100K 时 93 tok/s；32K 单流下是调优过 vLLM 的 1.10 倍。 速度来自这几样：MTP-3 投机解码（带精确 p/q 验证，默认开）、--spec-draft-p-min 0、65,536…

Digest-only: incomplete methodology (hardware state, measurement method).

DGX Spark x1 × DeepSeek-V4.1-Flash

Sep 15, 2026 · by @TeksEdge

Context Quant Engine Decode Prefill
— — custom experimental engine 17.1 tok/s —
— — custom experimental engine 24.3 tok/s —
— — custom experimental engine 28.2 tok/s —
— — custom experimental engine 31.8 tok/s —
— — custom experimental engine 36.6 tok/s —

So yes, this is another DeepSeek wave! DeepSeek-V4.1-Flash is roughly 510GB on disk. One DGX Spark has only 128GB of unified memory. Yet @0xBakeer (in replies) now has an experimental build running it at: 📝 17.1 tok/s prose 🐍 24.3 tok/s Python 🟨 28.2 tok/s JavaScript 🗄️ 31.8 tok/s SQL 🎮 36.6 tok/s HTML game How is this done? DeepSeek has 384 routed experts/layer but activates only 6 per token. So instead of treating…

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, context length, measurement method).

Unspecified platform × Qwen3.8 Flash-Next

Sep 15, 2026 · by @wei_wang

Context Quant Engine Decode Prefill
— EXL3 K4.25 + PLE FP8 qwen38-exl3-ple8 196.7 tok/s —
— EXL3 K4.25 + PLE FP8 qwen38-exl3-ple8 278.0 tok/s —
— EXL3 K4.25 + PLE FP8 qwen38-exl3-ple8 444.4 tok/s —
— EXL3 K4.25 + PLE FP8 qwen38-exl3-ple8 682.9 tok/s —
— EXL3 K4.25 + PLE FP8 qwen38-exl3-ple8 1020.0 tok/s —

花了一些时间来优化本地部署的 Qwen3.8 Flash-Next。Prose 解码跑到了单路 196.7 tok/s、16 路并发 1020 tok/s，这个成绩确实惊喜。 我们目前的本地定制配置如下： - EXL3 K4.25 专家权重，PLE FP8 表通过 NVMe mmap 读取，32 个读取线程，预读上限 2048 段。 - 主模型与 MTP 共享输出头，权重采用行级 FP8 E4M3＋FP32 缩放。投影矩阵行数 ≤128 时走 W8A16，更大行数走 W8A8。GDN 投影保留 BF16。 - 输出头节省约 0.59 GiB 显存，KV 预算增加 512 MiB，总池从 880600 扩到 911141 tokens。 - FP8 KV、MTP3、16 个调度槽，单请求上下文上限 262144，max_num_batched_tokens=2048。显式 KV 预算为 16204547298 bytes。…

Digest-only: hardware.name missing; incomplete methodology (engine+version, context length, measurement method).

Unspecified platform × Qwen3.8-27B

Sep 15, 2026 · by @Oluwaphilemon1 · third-party report

Context Quant Engine Decode Prefill
— Q4_K_M llama.cpp 20.0 tok/s —

Qwen3.8-27B might have a new answer to its biggest local-inference problem: overthinking. UkisAI built Swift-Qwen3.8-27B specifically around this idea. The motivation comes from research showing that reasoning models can sometimes reach the correct answer, continue thinking anyway, and eventually reason themselves away from it. So instead of simply telling Qwen3.8 to “think less,” the approach targets…

Digest-only: confidence 0.35 below 0.60; hardware.name missing; incomplete methodology (engine+version, context length, hardware state, measurement method).

DGX Spark × Qwen3.8-Flash-Next

Sep 15, 2026 · by @ViC305

Context Quant Engine Decode Prefill
262144 EXL3 4.05 bpw exllamav2 58.8 tok/s —
262144 EXL3 4.05 bpw vllm vllm-exl3 157.6 tok/s —
262144 EXL3 4.05 bpw vllm vllm-exl3 155.6 tok/s —
262144 EXL3 4.05 bpw vllm vllm-exl3 90.5 tok/s —
262144 EXL3 4.05 bpw vllm vllm-exl3 102.9 tok/s —
262144 EXL3 4.05 bpw vllm vllm-exl3 50.1 tok/s —
262144 EXL3 4.05 bpw vllm vllm-exl3 49.7 tok/s —
262144 EXL3 4.05 bpw vllm vllm-exl3 41.1 tok/s —
262144 EXL3 4.05 bpw vllm vllm-exl3 47.9 tok/s —

Qwen3.8-Flash-Next EXL3 just got a BIG one-Spark update. 🔥 58.8 tok/s single-stream through native ExLlamaV3.🚀 157.6 tok/s aggregate through vLLM across 8 streams.🤯 FULL 262,144 context on ONE DGX Spark. 4.05 bpw EXL3 pack. A much better serving envelope. 𝗕𝗘𝗙𝗢𝗥𝗘 → 𝗡𝗢𝗪 Previous public headline: 47.6 tok/s greedy p50 64K configured context MTP k=2 Concurrency not characterized Now: Native ExLlamaV3: 58.8 tok/s single…

Digest-only: incomplete methodology (engine+version, hardware state).

DGX Spark × Qwen3.8-27B

Sep 15, 2026 · by @Oluwaphilemon1 · third-party report

Context Quant Engine Decode Prefill
— NVFP4 vllm 23.7 tok/s —
— NVFP4 llama.cpp 38.8 tok/s —

Qwen3.8-27B has a very different idea of what “one-shot” means. This guy put Qwen3.8-27B and Qwen3.6-27B on the same voxel pagoda prompt. Same DGX Spark. Same NVFP4 precision. One attempt each. The results were almost comically different. Qwen3.8-27B spent 75,291 tokens and took 52 minutes 56 seconds. Qwen3.6-27B used 12,117 tokens and finished in 5 minutes 13 seconds. The strangest part? Qwen3.8 reportedly spent…

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, context length, hardware state, measurement method).

RTX 3090

Sep 15, 2026 · by @sudoingX

Context Quant Engine Decode Prefill
— Q4_K_M — 51.0 tok/s —
— — — 52.8 tok/s —
— Q4_K_M — 62.0 tok/s —
— — — 52.9 tok/s —
— — — 68.9 tok/s —

a 3090 capped at 250w does 51 tok/s with the flag. the same card at 275w does 62 tok/s. one slider, +22%, for 25 watts. the same contributor tested two quants on that card the same day, Q4_K_M against Dynamic 3.0: 52.8 tok/s vs 52.9 tok/s, a tie. the power limit moved the number more than the file did. if you are undervolting a rtx 3090 for local llms because a forum told you to in 2022, you are paying for that…

Digest-only: model.name missing; incomplete methodology (engine+version, context length).

RTX 4090 × Qwen 3.8 27B

Sep 15, 2026 · by @yume_arasaki

Context Quant Engine Decode Prefill
— int4 (output head only) vllm pinned version (exact version not specified, but… — —
38K int4 (output head only) vllm pinned version (exact version not specified, but… 141.2 tok/s —
38K int4 (output head only) vllm pinned version (exact version not specified, but… 133.9 tok/s —

I wanted to see how far I could push my RTX 4090 on Qwen 3.8 27B. So I treated it like a drag racing car. Stripped it, tuned it, pointed it down the strip. The results are a bit nuts. The 4090 turns four years old next month. It launched at $1,599 in October 2022, back when people bought it to play Cyberpunk. Today it's running a 27B parameter model at speeds that beat brand new hardware, and I don't think we've…

Full methodology stated (engine, quantization, context, hardware state, method) — 1 added to the benchmark board (1 of 3 measured rows queued; the rest are published here only).

DGX Spark × GLM-5.3-Flash

Sep 14, 2026 · by @JASONMCNAB

Context Quant Engine Decode Prefill
512 EXL3 2.05 bpw vllm sha256:905c02933be6021301db2dc284e24e3727467aa3a… 25.3 tok/s —
40K EXL3 2.05 bpw vllm sha256:905c02933be6021301db2dc284e24e3727467aa3a… 35.5 tok/s —
512 EXL3 2.05 bpw vllm sha256:905c02933be6021301db2dc284e24e3727467aa3a… — 327.0 tok/s
40K EXL3 2.05 bpw vllm sha256:905c02933be6021301db2dc284e24e3727467aa3a… — 773.0 tok/s
262144 EXL3 2.05 bpw vllm sha256:905c02933be6021301db2dc284e24e3727467aa3a… 64.1 tok/s —
262144 EXL3 2.05 bpw vllm sha256:905c02933be6021301db2dc284e24e3727467aa3a… 25.1 tok/s —
262144 EXL3 2.05 bpw vllm sha256:905c02933be6021301db2dc284e24e3727467aa3a… 41.0 tok/s —
262144 EXL3 2.05 bpw vllm sha256:905c02933be6021301db2dc284e24e3727467aa3a… 181.9 tok/s —
8002 EXL3 2.05 bpw vllm sha256:905c02933be6021301db2dc284e24e3727467aa3a… — 786.4 tok/s
16002 EXL3 2.05 bpw vllm sha256:905c02933be6021301db2dc284e24e3727467aa3a… — 822.1 tok/s
100002 EXL3 2.05 bpw vllm sha256:905c02933be6021301db2dc284e24e3727467aa3a… — 845.7 tok/s

GLM-5.3-Flash EXL3-2.05 on one NVIDIA Spark. Same @gitcommit90 recipe. Median decode (5 runs each, thinking off): ~512 in: 25.3 tok/s ~40k in: 35.5 tok/s Prefill median: 327 tok/s at ~512, 773 tok/s at ~40k. Ran at home. No extra tuning. Recipe: https://t.co/0CO7FGL6om What single-stream median did you keep at ~512 in on the 2.05 path?

Full methodology stated (engine, quantization, context, hardware state, method) — 2 added to the benchmark board, 1 already recorded (3 of 11 measured rows queued; the rest are published here only).

Also in this window

The 12 most recent posts are shown in full above; 11 posts from the same window are listed here in short form, each with its source post.

How to read this page

This digest is an automated transcription of public posts. Runs with a stated engine build, quantization, context length, hardware state and measurement method are published straight to the benchmark board with a link to their source; the rest stay here. These are author-reported numbers, not an independent DiggTop re-measurement: every row carries its provenance so you can check the original, and a number that turns out to be misreported is corrected or removed. Our own testing follows the methodology. Spotted an error? Reply on the source post or submit a corrected run at /submit.

Sources

FAQ

Are the numbers in this digest sourced?

They are transcribed from public X posts and the repositories those posts link to, with attribution and a source link on every entry. Rows that state the full methodology are published to the benchmark board automatically, so treat them as author-reported until a dispute is raised: the source link on every row is the check.

What does it take for a row to reach the sourced tier?

Engine and exact version, quantization, context length, hardware state and a described measurement method. Posts that state all five are published to the benchmark board as soon as the collector reads them, each row carrying its source link — nothing waits in a human review queue. Anything missing a condition stays in this digest.

Where does the data come from?

Public X posts about local-model inference measured on owned hardware, discovered by keyword search every two hours and linked back to the original thread. The digest is regenerated daily.

↑