Qwen · Qwen3 · 600M parameters

Qwen3 0.6B VRAM requirements

Qwen3 0.6B has 28 layers and uses grouped-query attention (8 KV heads). At Q4_K_M the weights come to 0.4 GB, and the best quantisation that fits a 24 GB card is FP16 / BF16.

Model

Hardware

Quantisation

Context 8K

KV cache precision

Runtime

Batch

GPUs

Monitor plugged into this GPU

Runs comfortably

2.0 GB of 21.8 GB · 9%

022 GB

Weights 0.4 GB

KV cache 0.9 GB

Runtime overhead 0.8 GB

Qwen3 0.6B at Q4_K_M leaves 19.8 GB spare on a RTX 4090. There is room to raise the context length or move up a quantisation level.

Generation679tok/s

Prompt processing57750tok/s

Max context32Ktokens

KV per 1K tokens0GB

Every quantisation of Qwen3 0.6B on a RTX 4090

Highlighted row is the highest quality that still fits at 8K context.

Quantisation	bpw	Weights	Total	Verdict	Max ctx	tok/s
FP16 / BF16	16.00	1.1 GB	2.8 GB	Runs comfortably	32K	365
FP8 (E4M3)	8.00	0.6 GB	2.2 GB	Runs comfortably	32K	551
INT8 / W8A8	8.50	0.6 GB	2.2 GB	Runs comfortably	32K	551
Q8_0 (GGUF)	8.50	0.6 GB	2.2 GB	Runs comfortably	32K	551
AWQ 4-bit	4.25	0.5 GB	2.2 GB	Runs comfortably	32K	576
GPTQ 4-bit	4.25	0.5 GB	2.2 GB	Runs comfortably	32K	576
MXFP4	4.25	0.5 GB	2.2 GB	Runs comfortably	32K	576
Q6_K	6.56	0.5 GB	2.1 GB	Runs comfortably	32K	606
Q5_K_M	5.67	0.4 GB	2.0 GB	Runs comfortably	32K	646
Q5_K_S	5.52	0.4 GB	2.0 GB	Runs comfortably	32K	654
Q4_K_M	4.85	0.4 GB	2.0 GB	Runs comfortably	32K	679
Q4_K_S	4.58	0.3 GB	2.0 GB	Runs comfortably	32K	690
Q4_0	4.55	0.3 GB	2.0 GB	Runs comfortably	32K	692
IQ4_XS	4.25	0.3 GB	2.0 GB	Runs comfortably	32K	704
Q3_K_M	3.91	0.3 GB	1.9 GB	Runs comfortably	32K	719
IQ3_M	3.70	0.3 GB	1.9 GB	Runs comfortably	32K	728
IQ3_XXS	3.06	0.3 GB	1.9 GB	Runs comfortably	32K	758
Q2_K	2.63	0.2 GB	1.9 GB	Runs comfortably	32K	780
IQ2_XXS	2.06	0.2 GB	1.8 GB	Runs comfortably	32K	811
IQ1_M	1.75	0.2 GB	1.8 GB	Runs comfortably	32K	829

Qwen3 0.6B on each GPU

Q4_K_M weights at 8K context, single card, monitor attached.

GPU	VRAM	GB/s	Verdict	Max ctx	tok/s
H100 SXM 80GB	80	3350	Runs comfortably	32K	1985
A100 80GB	80	2039	Runs comfortably	32K	1273
RTX 5090	32	1792	Runs comfortably	32K	1200
RTX 5080	16	960	Runs comfortably	32K	709
RTX 4090	24	1008	Runs comfortably	32K	679
RTX 5070 Ti	16	896	Runs comfortably	32K	667
RTX 3090	24	936	Runs comfortably	32K	661
Radeon RX 7900 XTX	24	960	Runs comfortably	32K	621
L40S	48	864	Runs comfortably	32K	592
RTX A6000	48	768	Runs comfortably	32K	553
RTX 3080 10GB	10	760	Runs comfortably	32K	548
Mac Studio M3 Ultra 256GB	256	819	Runs comfortably	32K	545
RTX 5070	12	672	Runs comfortably	32K	515
RTX 4080 Super	16	736	Runs comfortably	32K	512
RTX 4070 Ti Super	16	672	Runs comfortably	32K	471
Mac Studio M4 Max 128GB	128	546	Runs comfortably	32K	415
RTX 4070 Super	12	504	Runs comfortably	32K	360
RTX 4070	12	504	Runs comfortably	32K	360
RTX 5060 Ti 16GB	16	448	Runs comfortably	32K	353
Arc B580	12	456	Runs comfortably	32K	280
RTX 3060 12GB	12	360	Runs comfortably	32K	273
NVIDIA DGX Spark (GB10)	128	273	Runs comfortably	32K	220
Mac Mini M4 Pro 48GB	48	273	Runs comfortably	32K	215
RTX 4060 Ti 16GB	16	288	Runs comfortably	32K	211
Ryzen AI Max+ 395 128GB	128	256	Runs comfortably	32K	166

Architecture

Parameters	600M
Layers	28
Hidden size	1024
Attention heads / KV heads	16 / 8
Head dimension	128
Vocabulary	151,936
Trained context	32K
KV cache per 1K tokens	0 GB
Hugging Face	Qwen/Qwen3-0.6B

The Qwen family

Alibaba's series, and the broadest size ladder available — Qwen3 runs from 0.6B to 32B dense, plus 30B-A3B and 235B-A22B as mixture-of-experts. The 152k vocabulary makes the embedding table a large share of a small model's file. Qwen2.5-Coder is the same architecture trained for code.

huggingface.co/Qwen · qwenlm.github.io · all 14 Qwen models

Qwen3 1.7B
1.7B · 28 layers Qwen3 4B
4.0B · 36 layers Qwen3 8B
8.2B · 36 layers Qwen3 14B
14.8B · 40 layers Qwen3 32B
32.8B · 64 layers Qwen3 30B-A3B
30.5B · 3.3B active · 48 layers Qwen3 235B-A22B
235.1B · 22.1B active · 94 layers Qwen2.5 7B
7.6B · 28 layers

Direct answers

Qwen3 0.6B on RTX 5090
32 GB Qwen3 0.6B on RTX 4090
24 GB Qwen3 0.6B on RTX 3090
24 GB Qwen3 0.6B on RTX 5080
16 GB Qwen3 0.6B on RTX 5070 Ti
16 GB Qwen3 0.6B on RTX 5070
12 GB Qwen3 0.6B on RTX 4070 Ti Super
16 GB Qwen3 0.6B on RTX 4070 Super
12 GB Qwen3 0.6B on RTX 4070
12 GB