Qwen on AMD RDNA3
Can I run Qwen2.5 72B on an Radeon RX 7900 XTX?
Not at Q4_K_M — it needs 44.6 GB against 21.8 GB available. You would need 3 of these cards.
Won't fit
44.6 GB of 21.8 GB · 205%046 GB
Weights
41.2 GB
KV cache
2.5 GB
Runtime overhead
0.9 GB
Over the limit
22.9 GB
Short by 22.9 GB. You can run it with 35 of 80 layers on the Radeon RX 7900 XTX and the rest in system RAM, at roughly 1.44 tok/s — usable for batch work, painful for chat. A smaller quantisation or a shorter context is usually the better trade.
Generation1.44tok/s
Prompt processing355tok/s
Max context0tokens
KV per 1K tokens0GB
Every quantisation of Qwen2.5 72B on a Radeon RX 7900 XTX
Highlighted row is the highest quality that still fits at 8K context.
| Quantisation | bpw | Weights | Total | Verdict | Max ctx | tok/s |
|---|---|---|---|---|---|---|
| FP16 / BF16 | 16.00 | 135.4 GB | 138.8 GB | Won't fit | — | 0.30 |
| INT8 / W8A8 | 8.50 | 71.4 GB | 74.8 GB | Won't fit | — | 0.65 |
| Q8_0 (GGUF) | 8.50 | 71.4 GB | 74.8 GB | Won't fit | — | 0.65 |
| FP8 (E4M3) | 8.00 | 67.7 GB | 71.1 GB | Won't fit | — | 0.70 |
| Q6_K | 6.56 | 55.5 GB | 58.9 GB | Won't fit | — | 0.92 |
| Q5_K_M | 5.67 | 48.0 GB | 51.4 GB | Won't fit | — | 1.13 |
| Q5_K_S | 5.52 | 46.7 GB | 50.1 GB | Won't fit | — | 1.19 |
| Q4_K_M | 4.85 | 41.2 GB | 44.6 GB | Won't fit | — | 1.44 |
| AWQ 4-bit | 4.25 | 39.4 GB | 42.8 GB | Won't fit | — | 1.57 |
| GPTQ 4-bit | 4.25 | 39.4 GB | 42.8 GB | Won't fit | — | 1.57 |
| MXFP4 | 4.25 | 39.4 GB | 42.8 GB | Won't fit | — | 1.57 |
| Q4_K_S | 4.58 | 39.0 GB | 42.4 GB | Won't fit | — | 1.59 |
| Q4_0 | 4.55 | 38.8 GB | 42.2 GB | Won't fit | — | 1.60 |
| IQ4_XS | 4.25 | 36.3 GB | 39.7 GB | Won't fit | — | 1.81 |
| Q3_K_M | 3.91 | 33.6 GB | 36.9 GB | Won't fit | — | 2.09 |
| IQ3_M | 3.70 | 31.8 GB | 35.2 GB | Won't fit | — | 2.37 |
| IQ3_XXS | 3.06 | 26.6 GB | 30.0 GB | Won't fit | — | 3.63 |
| Q2_K | 2.63 | 23.1 GB | 26.5 GB | Won't fit | — | 5.62 |
| IQ2_XXS | 2.06 | 18.4 GB | 21.8 GB | Won't fit | 8K | 23.3 |
| IQ1_M | 1.75 | 15.9 GB | 19.3 GB | Runs comfortably | 16K | 31.4 |