How to read the DiggTop board
Board state as of 2026-10-07: 468 rows across 70 GPUs and 165 models — 336 sourced (334 attributed claims plus 2 measurements of our own) and 132 partial. (The model count went down from 239 since September: duplicate entities were merged, so the same model is no longer split across two pages.) Every row carries a source link. Here is what the badges mean, because a number is only useful if you know where it came from.
The three marks
| Mark | Means |
|---|---|
| ✓ | Sourced. The row states engine and version, quantisation, context length, a specific hardware configuration, and it links to the author's original page. DiggTop does not re-run other people's measurements. |
| self | Our own measurements (self-measured). A DiggTop team run, with the parameters recorded in the row. |
| … | Partial. One of those fields is missing — often the engine version, which the source platform never reported. Partial rows are visible, marked, and excluded from every aggregate. |
Why not every row is sourced
The tempting move is to mark every imported run as sourced so the board looks bigger. We don't, because that turns someone else's number into our endorsement — and the whole point of the site is that the numbers are traceable.
There is a harder version of the problem. 114 of the partial rows have no engine version on the source platform — not "we didn't look", the field is empty upstream and has stayed empty. Our checklist requires the engine version, so those rows can never reach ✓. They stay visible (you can see the hardware, the model, the quant) but they are marked partial rather than quietly promoted.
What "sourced" does and doesn't mean
- It means the provenance and method are checkable, not that the number is reproducible on your machine.
- It is measured on one model at one context length. A card's "best tok/s" is on some specific model, quant and context — check those columns before comparing.
- Imported rows are still useful; they are just labelled as imported. Treat the mark as a statement about what the source recorded, not as a judgement about the hardware.
Adding your own run
If the model or GPU you care about is missing — and plenty are, new MoE releases especially — /submit issues a ticket and one prompt. Your local model measures and posts three steps; nothing is installed; the row is labelled sourced if it states the five fields, and partial if it does not. Agents can do the same through the documented API (see /llms.txt).
FAQ
What does the checkmark mean?
It means the row states engine and version, quantisation, context length, a specific hardware configuration, and a link to the author's original page — the bar our methodology sets. DiggTop does not re-run other people's measurements, so it is a statement about the data's provenance, not a guarantee of the speed on your machine.
Why do some imported runs stay partial?
Because the source platform does not report the engine version for them, and our checklist requires it. 114 of the 132 partial rows are in this state. We keep them visible but marked partial instead of promoting them.
Can I trust a run without a checkmark?
Read it as a community data point with a source link. It tells you what one person measured on one setup; it is not our endorsement. Prefer sourced rows when you are comparing, and use the partial ones for breadth.
How do I submit my own run?
Go to /submit, copy the single prompt, and send it to your local model. It measures and posts three steps back to the page, you confirm, and the run is added to the board.
Sources
- Board: /benchmarks · methodology
- Review policy detail:
docs/data-review-2026-09-09.md