NVIDIA · Blackwell

What runs on a RTX PRO 6000 Blackwell

96 GB at 1792 GB/s. The largest model that fits at Q4_K_M is gpt-oss 120B-A5.1B, at roughly 216 tokens per second.

Runs comfortably

6.4 GB of 89.4 GB · 7%
089 GB
Weights 4.6 GB
KV cache 1.0 GB
Runtime overhead 0.8 GB

Llama 3.1 8B at Q4_K_M leaves 83.0 GB spare on a RTX PRO 6000 Blackwell. There is room to raise the context length or move up a quantisation level.

Generation222tok/s
Prompt processing6590tok/s
Max context128Ktokens
KV per 1K tokens0GB

Every quantisation of Llama 3.1 8B on a RTX PRO 6000 Blackwell

Highlighted row is the highest quality that still fits at 8K context.

QuantisationbpwWeights TotalVerdictMax ctxtok/s
FP16 / BF16 16.00 15.0 GB 16.8 GB Runs comfortably 128K 75.5
INT8 / W8A8 8.50 7.7 GB 9.5 GB Runs comfortably 128K 140
Q8_0 (GGUF) 8.50 7.7 GB 9.5 GB Runs comfortably 128K 140
FP8 (E4M3) 8.00 7.5 GB 9.3 GB Runs comfortably 128K 144
Q6_K 6.56 6.1 GB 8.0 GB Runs comfortably 128K 173
AWQ 4-bit 4.25 5.4 GB 7.2 GB Runs comfortably 128K 193
GPTQ 4-bit 4.25 5.4 GB 7.2 GB Runs comfortably 128K 193
MXFP4 4.25 5.4 GB 7.2 GB Runs comfortably 128K 193
Q5_K_M 5.67 5.3 GB 7.1 GB Runs comfortably 128K 196
Q5_K_S 5.52 5.2 GB 7.0 GB Runs comfortably 128K 201
Q4_K_M 4.85 4.6 GB 6.4 GB Runs comfortably 128K 222
Q4_K_S 4.58 4.4 GB 6.2 GB Runs comfortably 128K 231
Q4_0 4.55 4.4 GB 6.2 GB Runs comfortably 128K 232
IQ4_XS 4.25 4.1 GB 5.9 GB Runs comfortably 128K 244
Q3_K_M 3.91 3.8 GB 5.7 GB Runs comfortably 128K 259
IQ3_M 3.70 3.7 GB 5.5 GB Runs comfortably 128K 269
IQ3_XXS 3.06 3.2 GB 5.0 GB Runs comfortably 128K 305
Q2_K 2.63 2.8 GB 4.6 GB Runs comfortably 128K 335
IQ2_XXS 2.06 2.3 GB 4.2 GB Runs comfortably 128K 385
IQ1_M 1.75 2.1 GB 3.9 GB Runs comfortably 128K 420

Models on a RTX PRO 6000 Blackwell

Q4_K_M at 8K context. Bandwidth sets the speed; capacity sets the ceiling.

ModelParamsWeightsVerdictMax ctxtok/s
DeepSeek-R1 671B-A37B 671B 379.0 GB Won't fit — 2.52
Qwen3 235B-A22B 235.1B 132.8 GB Won't fit — 7.90
gpt-oss 120B-A5.1B 116.8B 66.0 GB Runs comfortably 128K 216
Llama 4 Scout 109B-A17B 109B 61.7 GB Runs comfortably 143K 93.5
Qwen2.5 72B 72.7B 41.2 GB Runs comfortably 128K 27.5
Llama 3.1 70B 70.5B 40.0 GB Runs comfortably 128K 28.3
Mixtral 8x7B 46.7B 26.4 GB Runs comfortably 32K 120
Command R 35B 35.0B 20.1 GB Runs comfortably 55K 46.7
Yi-1.5 34B 34.4B 19.5 GB Runs comfortably 32K 56.8
Qwen3 32B 32.8B 18.6 GB Runs comfortably 128K 59.0
Qwen2.5-Coder 32B 32.8B 18.6 GB Runs comfortably 128K 59.0
DeepSeek-R1-Distill-Qwen 32B 32.8B 18.6 GB Runs comfortably 128K 59.0
Qwen3 30B-A3B 30.5B 17.3 GB Runs comfortably 128K 159
Gemma 3 27B 27.4B 15.6 GB Runs comfortably 128K 70.7
Mistral Small 24B 23.6B 13.4 GB Runs comfortably 32K 82.7
gpt-oss 20B-A3.6B 20.9B 11.9 GB Runs comfortably 128K 311
Qwen3 14B 14.8B 8.5 GB Runs comfortably 128K 126
Phi-4 14B 14.7B 8.4 GB Runs comfortably 16K 125
Mistral NeMo 12B 12.3B 7.0 GB Runs comfortably 128K 149
Gemma 3 12B 12.2B 7.0 GB Runs comfortably 128K 151
GLM-4 9B 9.4B 5.4 GB Runs comfortably 128K 203
Qwen3 8B 8.2B 4.7 GB Runs comfortably 128K 214
Llama 3.1 8B 8.0B 4.6 GB Runs comfortably 128K 222
DeepSeek-R1-Distill-Llama 8B 8.0B 4.6 GB Runs comfortably 128K 222
Qwen2.5 7B 7.6B 4.4 GB Runs comfortably 128K 247
Mistral 7B v0.3 7.3B 4.1 GB Runs comfortably 32K 245
Gemma 3 4B 4.3B 2.5 GB Runs comfortably 128K 405
Qwen3 4B 4.0B 2.3 GB Runs comfortably 128K 380
Llama 3.2 3B 3.2B 1.8 GB Runs comfortably 128K 478
Llama 3.2 1B 1.2B 0.7 GB Runs comfortably 128K 1235

Direct answers

Llama 3.1 8B on RTX PRO 6000 Blackwell
8.0BLlama 3.1 70B on RTX PRO 6000 Blackwell
70.5BLlama 3.2 3B on RTX PRO 6000 Blackwell
3.2BLlama 3.2 1B on RTX PRO 6000 Blackwell
1.2BLlama 4 Scout 109B-A17B on RTX PRO 6000 Blackwell
109BQwen3 8B on RTX PRO 6000 Blackwell
8.2BQwen3 14B on RTX PRO 6000 Blackwell
14.8BQwen3 32B on RTX PRO 6000 Blackwell
32.8B