AMD · RDNA3

What runs on a Radeon RX 7900 XT

20 GB at 800 GB/s. The largest model that fits at Q4_K_M is Gemma 3 27B, at roughly 27.6 tokens per second.

Model

Hardware

Quantisation

Context 8K

KV cache precision

Runtime

Batch

GPUs

Monitor plugged into this GPU

Runs comfortably

6.4 GB of 18.0 GB · 36%

018 GB

Weights 4.6 GB

KV cache 1.0 GB

Runtime overhead 0.8 GB

Llama 3.1 8B at Q4_K_M leaves 11.6 GB spare on a Radeon RX 7900 XT. There is room to raise the context length or move up a quantisation level.

Generation87.5tok/s

Prompt processing2694tok/s

Max context101Ktokens

KV per 1K tokens0GB

Every quantisation of Llama 3.1 8B on a Radeon RX 7900 XT

Highlighted row is the highest quality that still fits at 8K context.

Quantisation	bpw	Weights	Total	Verdict	Max ctx	tok/s
FP16 / BF16	16.00	15.0 GB	16.8 GB	Fits, but tight	18K	29.3
INT8 / W8A8	8.50	7.7 GB	9.5 GB	Runs comfortably	76K	54.9
Q8_0 (GGUF)	8.50	7.7 GB	9.5 GB	Runs comfortably	76K	54.9
FP8 (E4M3)	8.00	7.5 GB	9.3 GB	Runs comfortably	78K	56.4
Q6_K	6.56	6.1 GB	8.0 GB	Runs comfortably	88K	67.7
AWQ 4-bit	4.25	5.4 GB	7.2 GB	Runs comfortably	94K	75.9
GPTQ 4-bit	4.25	5.4 GB	7.2 GB	Runs comfortably	94K	75.9
MXFP4	4.25	5.4 GB	7.2 GB	Runs comfortably	94K	75.9
Q5_K_M	5.67	5.3 GB	7.1 GB	Runs comfortably	95K	77.3
Q5_K_S	5.52	5.2 GB	7.0 GB	Runs comfortably	96K	79.2
Q4_K_M	4.85	4.6 GB	6.4 GB	Runs comfortably	101K	87.5
Q4_K_S	4.58	4.4 GB	6.2 GB	Runs comfortably	102K	91.4
Q4_0	4.55	4.4 GB	6.2 GB	Runs comfortably	102K	91.8
IQ4_XS	4.25	4.1 GB	5.9 GB	Runs comfortably	104K	96.6
Q3_K_M	3.91	3.8 GB	5.7 GB	Runs comfortably	107K	103
IQ3_M	3.70	3.7 GB	5.5 GB	Runs comfortably	108K	107
IQ3_XXS	3.06	3.2 GB	5.0 GB	Runs comfortably	112K	122
Q2_K	2.63	2.8 GB	4.6 GB	Runs comfortably	115K	134
IQ2_XXS	2.06	2.3 GB	4.2 GB	Runs comfortably	119K	155
IQ1_M	1.75	2.1 GB	3.9 GB	Runs comfortably	121K	170

Q4_K_M at 8K context. Bandwidth sets the speed; capacity sets the ceiling.

Model	Params	Weights	Verdict	Max ctx	tok/s
DeepSeek-R1 671B-A37B	671B	379.0 GB	Won't fit	—	1.75
Qwen3 235B-A22B	235.1B	132.8 GB	Won't fit	—	2.99
gpt-oss 120B-A5.1B	116.8B	66.0 GB	Won't fit	—	14.5
Llama 4 Scout 109B-A17B	109B	61.7 GB	Won't fit	—	4.52
Qwen2.5 72B	72.7B	41.2 GB	Won't fit	—	1.26
Llama 3.1 70B	70.5B	40.0 GB	Won't fit	—	1.32
Mixtral 8x7B	46.7B	26.4 GB	Won't fit	—	10.0
Command R 35B	35.0B	20.1 GB	Won't fit	—	2.14
Yi-1.5 34B	34.4B	19.5 GB	Won't fit	—	6.02
Qwen3 32B	32.8B	18.6 GB	Won't fit	—	7.32
Qwen2.5-Coder 32B	32.8B	18.6 GB	Won't fit	—	7.32
DeepSeek-R1-Distill-Qwen 32B	32.8B	18.6 GB	Won't fit	—	7.32
Qwen3 30B-A3B	30.5B	17.3 GB	Won't fit	—	74.6
Gemma 3 27B	27.4B	15.6 GB	Fits, but tight	14K	27.6
Mistral Small 24B	23.6B	13.4 GB	Runs comfortably	24K	32.2
gpt-oss 20B-A3.6B	20.9B	11.9 GB	Runs comfortably	113K	149
Qwen3 14B	14.8B	8.5 GB	Runs comfortably	56K	49.5
Phi-4 14B	14.7B	8.4 GB	Runs comfortably	16K	49.1
Mistral NeMo 12B	12.3B	7.0 GB	Runs comfortably	65K	58.7
Gemma 3 12B	12.2B	7.0 GB	Runs comfortably	128K	59.4
GLM-4 9B	9.4B	5.4 GB	Runs comfortably	128K	80.3
Qwen3 8B	8.2B	4.7 GB	Runs comfortably	89K	84.6
Llama 3.1 8B	8.0B	4.6 GB	Runs comfortably	101K	87.5
DeepSeek-R1-Distill-Llama 8B	8.0B	4.6 GB	Runs comfortably	101K	87.5
Qwen2.5 7B	7.6B	4.4 GB	Runs comfortably	128K	97.3
Mistral 7B v0.3	7.3B	4.1 GB	Runs comfortably	32K	96.8
Gemma 3 4B	4.3B	2.5 GB	Runs comfortably	128K	164
Qwen3 4B	4.0B	2.3 GB	Runs comfortably	106K	154
Llama 3.2 3B	3.2B	1.8 GB	Runs comfortably	128K	193
Llama 3.2 1B	1.2B	0.7 GB	Runs comfortably	128K	512