Run 70B Models on Two Used RTX 3090s
The single-GPU VRAM wall at 24GB is exactly what the 48GB dual-3090 build breaks. Two RTX 3090s (24GB each) hold a 70B model at Q4/K-quant entirely in VRAM — no CPU offload, no "it works but at 2 tok/s" surprise.
As of 2026-09-28. Used RTX 3090 24GB has moved from a $1,050 eBay-sold average in May 2026 (Howard Resource Group) to $1,300 in our latest weekly snapshot (2026-09-28); a dual-3090 board+card combo now sits closer to the $2,600 end of the $2,000–2,600 range seen through August 2026. Verify current listings before budgeting.
What fits, and at what speed
| Model | Quant | Weights | Fits 48GB? | Expect |
|---|---|---|---|---|
| Llama-3.1-70B | Q4_K_M | ~40 GB | yes, + KV | ~10–15 tok/s gen |
| Qwen3-32B | Q4_K_M | ~20 GB | yes, large context | ~25–35 tok/s gen |
| MoE 30B-A3B class | FP8/Q4 | ~20 GB | yes, huge context | high tok/s (small active params) |
Community-reported numbers on this exact configuration cluster around these ranges; per-run detail lives on our benchmarks board with attribution.
Setup notes
- llama.cpp: offload all layers (
-ngl 999), enable flash attention (-fa 1), and keep KV in GPU. Dual GPUs are used automatically when total VRAM is needed. - PCIe topology matters less than people think for inference: weights sit in VRAM after load; the CPU↔GPU path only matters at startup.
- Power: two 350W cards under load — a 1000W quality PSU is the floor.
- Cooling: two full-slot blower cards in a mid-tower will be loud; budget for a good case or blower-model cards.
When this is the wrong build
- If you only need 14B-or-smaller models, a single 12–16GB card is cheaper and simpler.
- If you need 100B+ or 200K contexts, look at the 48GB workstation tier (RTX A6000 48GB, used ≈ $2,200–2,800) or dual-5090 money — our 48GB build guide weighs the options.
- The used market moves fast; price checks older than a week are already stale (see our weekly price snapshot).
Sources
- Used 3090 pricing: Howard Resource Group — RTX 30 series used prices (eBay sold, 2026-05); current value from our 2026-09-28 snapshot
- A6000 48GB used/new range: SpecPicks A6000 48GB review
- Dual-3090 vs A6000 trade-off discussion: OpenClaw DC 48GB build comparison (2026-08)
- Methodology for any speed claims: our benchmark methodology
- DiggTop data: RTX 3090 · dual-3090 70B build · benchmarks board