One Spark, 552B: Engine Beats Card Count

A DGX Spark is a $3,999 desktop box with 128 GB of unified memory and a GB10 part in it. DeepSeek-V4.1-Flash is a 552B-parameter mixture-of-experts model that ships as a ~475 GiB checkpoint. The obvious read is that one cannot run the other, and the second obvious read is that you would need a stack of boxes if it could.

Our board says both intuitions are wrong in interesting ways. One Spark does run it — slowly, and only with an engine someone wrote for the job. Two Sparks run it very differently depending on which engine is driving them. And going from two boxes to four does not buy what you would expect.

As of 2026-09-15. Every number is an imported, sourced row on our board, with engine, format and context attached. We did not run these tests, and they come from different submitters on different days — see the caveat at the end.

What the board has, in one table

All rows: DeepSeek-V4.1-Flash (552B MoE) on DGX Spark hardware, decode tok/s.

Read the box column against the speed column and nothing lines up. One box reaches 47; two reach 154; three, with a more capable engine on paper, come in at 46; four land at 82. If card count were the variable, this table would be monotonic. It is not, because card count is not the variable.

The variable is the engine, and it is not close

Look at the single-box rows first, because they isolate it. The same 552B checkpoint on the same class of box:

• a custom v41_engine.py (a Triton-based build written by the submitter) produces 13–22 tok/s at 32K context, with one row at 47;
• EXL3 2.05bpw on a single Spark produces 64.1 tok/s — and it does that on a different, larger model (GLM-5.3-Flash), at a tiny 512-token context.

That is a 3–5× spread with the hardware held constant and the engine variable. The second row is also the more interesting one: EXL3's 2.05 bits per weight fits an enormous model into 128 GB of unified memory with enough left for a working context, and the kernel is fast enough to produce a usable number. Nothing about the box changed.

Two boxes: the best result on the board

The two-Spark rows are the fastest thing we have for this model, and they are also the most informative:

EXL3 2.9bpw, tensor-parallel across two boxes, with speculative decoding. At 4K context it is 3.3× the best single-box number, at 559K it is still 30.5 — and the 5× drop between 4K and 17K is the speculative-decoding acceptance curve, not a memory wall. We wrote about that curve separately in Context Is Not a Cliff; here what matters is that the fastest configuration on the board is the one that pairs the right format with the right engine — not the one with the most boxes.

Three and four boxes: more hardware, less throughput

The three-box row is the trap. It runs vLLM with EXL3 3.5bpw at a 300,000-token context and produces 46 tok/s. The four-box row, same engine, same format, same context, produces 82 — a 1.8× gain from adding a box, which is a poor return compared to what two boxes did with a different engine.

Two things are going on, and neither is "three boxes are slower than two":

1. The context is different. The two-box ladder at 4K is not competing with the four-box row at 300K; the comparable two-box numbers at long context are 32.4 (17K) and 30.5 (559K). At matching long context, four boxes on vLLM do beat two boxes on exl3 — 82 vs 35.
2. The format is different. EXL3 2.9bpw (two boxes) versus EXL3 3.5bpw (three and four boxes) is a different checkpoint, decompressed by a different kernel. As our format comparison shows, that alone has been worth 2× on this hardware family.

So the honest reading is not "add boxes, lose speed". It is: the four-box row is not comparable to the two-box row, and any table that puts them on the same line and sorts by tok/s is lying to you.

What this means if you are considering this class of machine

• Feasibility is a format question. A 552B MoE ran on one $3,999 box because someone shipped a 2.05 bpw quantization with a fast kernel. Before asking "how many cards", ask "what is the lowest-bit, best-supported format for my model".
• Then it is an engine question. Same box: 22 tok/s one way, 64 another. The engine's dequantization path is a bigger lever than the second GPU.
• Card count buys context, not speed. The two-box configuration's headline is not 154 tok/s — it is 154 tok/s at 4K and 30.5 at 559K. Scaling out buys you the ability to hold a huge window, and the per-token speed at long context is roughly what one well-engineered box would give you anyway.
• Check the context column before the speed column. It is the single most common way these tables mislead.
• We found and fixed a data-hygiene problem while writing this. The DGX Spark had been imported under four spellings (DGX Spark, DGX Spark (GB10), DGX Spark GB10, GB10 x2), which split its aggregates three ways and gave the multi-box rows wrong VRAM values (512 GB for a four-box row, against a per-card convention of 128). The entries are now merged — the single-box page carries 44 runs instead of 20 — and the old URLs redirect. Two pairs of rows with identical values stayed in: they come from different submitters, not from a double import.

FAQ

Can a single DGX Spark run a 552B-parameter model?

Yes, with the right quantization. Rows on our board show DeepSeek-V4.1-Flash running on one Spark at 13–47 tok/s at 8K–32K context, and a larger model at 64.1 tok/s under EXL3 2.05bpw.

Why is four boxes not four times faster?

Because throughput at this scale is set by the engine's dequantization and communication path, not by raw card count. The four-box vLLM rows (82 tok/s at 300K context) use a different format from the two-box EXL3 rows (154 tok/s at 4K), so the two configurations are not comparable.

What is the best configuration on the board for this model?

Two DGX Sparks with EXL3 2.9bpw and speculative decoding: 154 tok/s at 4K context, 45.5 at 1K, and 30.5 at 559K context.

How much does context length matter here?

At long context the two-box configuration settles at 30–35 tok/s regardless of whether the window is 17K or 559K. Decode throughput is largely insensitive to context once the speculative-decoding acceptance rate stabilises — see the context piece.

Is one box enough for a 500B-class model in practice?

For batch-1 chat at a few dozen tokens per second, yes — that is what the single-box rows show. For serving multiple users, the four-box rows at 300K context are the relevant shape, and there the limit is memory and communication rather than format.

Sources

• All rows: imported and sourced entries on the board with engine, format and context attached. Per-hardware views: DGX Spark · DeepSeek-V4.1-Flash · GLM-5.3-Flash.
• Related analysis: Context Is Not a Cliff · 3.5 Bits Beat 4 Bits · MoE Expert Offload · MoE Expert Cache.
• Full benchmark board · methodology.

Imported upstream measurements as of 2026-09-14, from multiple submitters on different days. This is not a controlled experiment: engines, formats and contexts differ between rows, which is precisely the point being made. Each row carries its provenance on the board.