Mixture-of-Experts
Big models with small active footprints — if the experts are placed well.
A mixture-of-experts model holds far more parameters than it uses per token, which moves the bottleneck from raw VRAM to expert placement: which experts stay resident, which stream from system RAM, and how much of the weights move per token. These pieces measure what that costs.
MoE Expert Cache: VRAM Stops Being the Ceiling
Expert caching moves cold MoE experts to system RAM — FreeToken paper plus llama.cpp RFC #24528. Community: 90+ tps on a 16GB 5070 Ti. As of 2026-09-07.
- Sep 11, 2026 · 9 min readMoE Expert Offload: Same Weights, 26× ApartSame weights, same box: 3.3 tok/s with whole-layer expert offload vs 150 with row-selective UVA reads, and what decides big-MoE speed on consumer GPUs.
- Sep 15, 2026 · 7 min readTwo RTX 3060s: 16 tok/s Dense, 94 tok/s MoEThe same dual-3060 rig runs a 30B dense model at 16 tok/s and a 30B mixture-of-experts at 94. That 6x spread decides whether the build is worth it.
- Sep 15, 2026 · 7 min readOne Spark, 552B: Engine Beats Card CountOne DGX Spark runs a 552B MoE at 47 tok/s; two reach 154, three drop to 46. At this scale the engine and format decide throughput, not the box count.
- Sep 28, 2026 · 9 min readOne 180B MoE, Five Formats: 41 Sourced RunsQwen3.8-Flash-Next's split entities merged into one page: 41 sourced rows across three checkpoints, 20.3 to 474 tok/s, and what fits a 24GB card.
- Oct 12, 2026 · 9 min readX local-model bench digest — Oct 12, 2026Local-LLM throughput measured on X: 40 posts and 109 runs, with hardware, quantization, engine, context and source links.
- Sep 12, 2026 · 9 min readX local-model bench digest — Sep 12, 2026Local-LLM throughput measured on X: 25 posts, 81 runs — hardware, quantization, engine and tok/s with a source link for every number.
- Oct 13, 2026 · 10 min readX local-model bench digest — Oct 13, 2026Local-LLM throughput measured on X: 33 posts and 89 runs, with hardware, quantization, engine, context and source links.
- Sep 13, 2026 · 9 min readX local-model bench digest — Sep 13, 2026Local-LLM throughput measured on X: 14 posts, 51 runs — hardware, quantization, engine and tok/s with a source link for every number.
- Oct 16, 2026 · 10 min readX local-model bench digest — Oct 16, 2026Local-LLM throughput measured on X: 27 posts, 54 runs — hardware, quantization, engine and tok/s with a source link for every number.
- Oct 9, 2026 · 10 min readX local-model bench digest — Oct 9, 2026Local-LLM throughput measured on X: 32 posts, 79 runs — hardware, quantization, engine and tok/s with a source link for every number.
- Sep 17, 2026 · 10 min readX local-model bench digest — Sep 17, 2026Local-LLM throughput measured on X: 30 posts and 68 runs, with hardware, quantization, engine, context and source links.