RTX 5090 vs RTX 3090 for Local LLMs

Data as of 2026-10-07 (sourced runs); prices are the 2026-09-28 snapshot from our weekly snapshot. Prices move weekly — treat dollar figures as a snapshot, not a promise.

TL;DR

What the community data says

DiggTop tracks attributed, methodology-linked runs. As of 2026-09-05:

Two caveats before you read the table. Both ceilings are small-model rows, so the headline gap (~3×) is about memory bandwidth, not about the models you probably care about. On the same model — Qwen3.8-27B, sourced rows only — the 5090's best is 435.1 tok/s against the 3090's 164.6 tok/s, a 2.6× gap, and the 3090's average on that model (100.1 across 7 rows) is dragged down by long-context runs at Q4. Second, the 5090's fastest 27B row uses NVFP4, Blackwell's native 4-bit format that Ampere simply does not have.

The 5090 row above is dominated by Blackwell-era runs (NVFP4/FP8). The 3090 has no FP4 path — its best quant is GGUF Q4_K_M or IQ4_XS.

VRAM math: 32GB vs 24GB

Local inference is decided by one number: does the model fit in VRAM, with room for KV cache and context?

For 70B-class models the answer is the same on both cards: you need 48GB. That means a pair of used 3090s ($2,000–2,600), a used A6000 48GB ($2,200–2,800), or the new RTX PRO 6000 Blackwell if you have the budget. See our 70B on two used 3090s guide.

The FP4 lever (5090 only)

NVIDIA moved 4-bit math into hardware with Blackwell. On a 5090, a 27B model at NVFP4 has roughly half the VRAM footprint of FP16 and runs on tensor cores built for it — that is why the fastest sourced 27B row on our board is 435.1 tok/s on a 5090 (IQ4_XS, 131K context). A faster 4,546.8 tok/s NVFP4/sglang row exists in the database but is still partial, so it is not a board result.

The 3090 gets no such help: everything above 8-bit runs through GGUF K-quants or IQ quants on CUDA cores. Efficient, proven, community-standard — but the 5090's 4-bit ceiling is in a different league for models that fit its 32GB.

Price and value

US retail stock of the 5090 has been thin enough that street prices roughly doubled between our 09-10 and 09-28 snapshots: $4,899.99 to $9,000 (about +350% over its $1,999 MSRP). The used 3090 moved the same way, from ~$1,050 to $1,300. At these numbers the used 3090 still buys about 5× the VRAM per dollar — see the full ranking on the value index.

Verdict

• Budget / value / 70B plans: used RTX 3090 — and consider two of them for 70B Q4.
• New build, 27B-and-under with maximum speed: RTX 5090, only if the $156/GB premium is acceptable. NVFP4 is the reason to pay it.
• 70B on a single card: neither. Budget for 48GB (used A6000 or dual 3090).
• Intel/AMD alternative: Arc Pro B60 24GB ($599, $25/GB) is the cheapest 24GB card tracked — no FP4, but unbeatable VRAM per dollar.

Sources

• DiggTop sourced runs: benchmarks board · head-to-head · value index
• Current prices: our 2026-09-28 snapshot ($9,000 new 5090, $1,300 used 3090)
• 5090 street-price context (TechSpot, Aug 2026, superseded by the snapshot above): 199it summary
• Used 3090 market history: Howard Resource Group
• A6000 used $2,200–2,800 / new ~$4,650: SpecPicks
• Arc Pro B60 24GB at $599 US retail: Chiphell

FAQ

Used 3090 or new 5090 for local LLMs?

Depends on your target models. The 3090: 24 GB around $1,300 used (2026-09-28 snapshot) — still the best $/GB we track and the default value pick. The 5090: 32 GB plus NVFP4, which opens model classes a 3090 cannot load, at roughly 7× the card price. For 70B-class models, neither is enough — budget 48 GB instead.

What does the community benchmark data show?

On shared models the 5090 wins tok/s (newer architecture, faster memory, NVFP4 on supported models), while the used 3090 wins on cost by a wide margin — about 5× the VRAM per dollar at the 2026-09-28 snapshot. The head-to-head page shows the sourced per-model numbers.

Is 32 GB a real step up from 24 GB?

Yes, but not for 70B: it fits 27 B-class models at Q4 with long-context headroom, and NVFP4 pulls in some MoE models the 3090 cannot load. A 70B does not fit on either card — that is a 48 GB decision (used A6000, dual 3090, or RTX PRO 6000).

Does buying a 3090 now make sense if I plan to upgrade?

Yes. PCIe is stable, the 3090 can become the second GPU in a 70B build later, and it resells well. If your budget is tight, buy the 24 GB card now and treat the 48 GB step as a separate, later decision — that is the path the buying guide recommends.

Prices are weekly-snapshot items. Re-check prices before buying.