The 48GB VRAM Local LLM Build
As of 2026-09-05. Prices from our weekly snapshot; tok/s from the DiggTop board.
TL;DR
| Route | Cost | Pros | Cons |
|---|---|---|---|
| 2× used RTX 3090 | ~$2,000–2,600 | Cheapest 48GB; faster for batch; parts everywhere | Two cards: PSU, cooling, spacing |
| Used A6000 48GB | ~$2,200–2,800 | One card, one slot, 300W, NVLink to 96GB | Older arch; slower per dollar on 27B |
Both run 70B-class models at Q4. The 48GB tier is the threshold where local LLMs stop being a toy and start replacing API subscriptions.
Why 48GB
70B-class models at Q4 need ~40–48GB of VRAM with usable context. 24GB cards run 27B comfortably but cannot fit 70B; 32GB (5090) still falls short. 48GB is the first tier that runs Qwen 70B / Llama 70B / DeepSeek-V4-class models locally with 8–16k context.
Our board's big-VRAM runs are dominated by rented or multi-card rigs. Those rows are listed below with sourced rows only (131 of the 468 rows on the board are partial, and a partial row is not a result):
| Hardware | Model | tok/s | Quant | Context |
|---|---|---|---|---|
| NVIDIA H200 SXM ×4 | MiniMax-M2.7 | 495.9 | FP8 | 200K |
| DGX Spark ×2 | Qwen3.8-Flash-Next (hibrid48) | 474.0 | NVFP4 (output head) | 262K |
| CMP 170HX ×4 | GLM-5.3-Flash | 264.6 | W4A16 | 262K |
| RTX 3090 ×2 | Qwen3.6-27B | 240.8 | AutoRound INT4 | 200K |
| RTX PRO 6000 Blackwell ×4 | DeepSeek-V4.1-Flash | 231.6 | FP4+FP8 native | 64K |
Not on this list: the 250.6 tok/s R9700 ×3 row that used to head it — that one is still partial. The cheapest 48GB path in the table is the RTX 3090 pair at 240.8 tok/s, which is the point of this guide.
Dual RTX 3090 build sheet
- Cards: 2× used RTX 3090 (~$1,300 each, 2026-09-28 snapshot) — total 48GB
- PSU: 1000W+ quality unit. Two 3090s can pull 350W each sustained with 500W+ transients; do not cheap out here
- Cooling: the 3090 dumps ~350W of heat per card. Spaced slots or a riser, positive case airflow, and accept fan noise under load — or repad/repaste used cards before installing
- PCIe: x8/x4 is enough for inference — weights load once into VRAM, then PCIe barely matters. No NVLink needed for tensor-parallel-inference splits on llama.cpp
- OS: Linux (Ubuntu); CUDA toolkit + llama.cpp or vLLM
Community result on this exact tier: 240.8 tok/s best sourced single-stream run on dual 3090s (Qwen3.6-27B, AutoRound INT4, vLLM, 200K context — 6 sourced rows on the board), plenty for interactive chat on models that fit.
The A6000 route
If you want one card instead of a project: a used A6000 48GB (~$2,200–2,800) is a single-slot 300W blower that runs 70B Q4 with context, quietly, and leaves NVLink open for a 96GB future. It is slower than a 5090 on 27B and lacks FP4, but nothing else puts 48GB in one card at this price.
Power and running cost
A 48GB dual-3090 rig burns roughly 500–700W while generating. At the US average (~$0.16/kWh) that is about $0.08–0.11 per hour of heavy use — trivial next to API costs if you generate hours per day, meaningful if you leave it idle 24/7 (then ~$2.6/day). Set power limits and idle downclocks; the cards do not care.
What you can run at 48GB
| Model class | Fits? | Typical quant |
|---|---|---|
| 27B | Yes, with very long context | Q6_K / Q8_0 |
| 70B | Yes, 8–16k context | Q4_K_M / IQ4 |
| 70B MoE (A3B active) | Yes, comfortably | Q4–Q6 |
| 120B+ | No — 96GB tier | — |
Check the compatibility matrix for model × VRAM-tier data, compare RTX 3090 vs RTX 5090 head-to-head, and see the value index for current $/GB before you buy.
Sources
- DiggTop runs and hardware stats: benchmarks · matrix · value
- Used 3090 ~$1,300 (2026-09-28 snapshot): prices
- Used 3090 market history (May 2026, ~$1,050): Howard Resource Group
- A6000 used $2,200–2,800: SpecPicks
- Related: Run a 70B on two used RTX 3090s · $1,500 sweet-spot build
Prices are weekly-snapshot items. Re-check prices before buying.