Index

Models

48 models across 11 families, grouped by release. Architecture is pulled from each repository's config, so the shapes here are the ones the weights actually have.

Llama

Meta's open-weight series, and the default target for most local tooling. Every Llama 3.x model uses grouped-query attention with 8 KV heads, so the cache stays modest even at 70B. Llama 4 moved to mixture-of-experts: Scout and Maverick occupy 109B and 400B of memory but read only 17B per token.

7 models · 1.2B–405.9B · huggingface.co/meta-llama · llama.com

Llama 3.1

3 · 8.0B–405.9B

Llama 3.1 405B
405.9B · 126 layers Llama 3.1 70B
70.5B · 80 layers Llama 3.1 8B
8.0B · 32 layers

Llama 3.2

2 · 1.2B–3.2B

Llama 3.2 3B
3.2B · 28 layers Llama 3.2 1B
1.2B · 16 layers

Llama 4

2 · 109B–400B

Llama 4 Maverick 400B-A17B
400B · 17B active · 48 layers Llama 4 Scout 109B-A17B
109B · 17B active · 48 layers

Qwen

Alibaba's series, and the broadest size ladder available — Qwen3 runs from 0.6B to 32B dense, plus 30B-A3B and 235B-A22B as mixture-of-experts. The 152k vocabulary makes the embedding table a large share of a small model's file. Qwen2.5-Coder is the same architecture trained for code.

14 models · 600M–235.1B · huggingface.co/Qwen · qwenlm.github.io

Qwen3

8 · 600M–235.1B

Qwen3 235B-A22B
235.1B · 22.1B active · 94 layers Qwen3 32B
32.8B · 64 layers Qwen3 30B-A3B
30.5B · 3.3B active · 48 layers Qwen3 14B
14.8B · 40 layers Qwen3 8B
8.2B · 36 layers Qwen3 4B
4.0B · 36 layers Qwen3 1.7B
1.7B · 28 layers Qwen3 0.6B
600M · 28 layers

Qwen2.5

4 · 7.6B–72.7B

Qwen2.5 72B
72.7B · 80 layers Qwen2.5 32B
32.8B · 64 layers Qwen2.5 14B
14.8B · 48 layers Qwen2.5 7B
7.6B · 28 layers

Qwen2.5-Coder

2 · 7.6B–32.8B

Qwen2.5-Coder 32B
32.8B · 64 layers Qwen2.5-Coder 7B
7.6B · 28 layers

DeepSeek

V3 and R1 share one 671B mixture-of-experts architecture with 37B active per token, and both use multi-head latent attention — a single compressed KV vector per layer instead of a full set of heads. That gives R1 roughly a fifth of the cache per token of a dense 70B. The R1 distills are ordinary Qwen and Llama models fine-tuned on R1 output, and they fit on one card.

5 models · 8.0B–671B · huggingface.co/deepseek-ai · deepseek.com

DeepSeek-V3

1 · 671B

DeepSeek-V3 671B-A37B
671B · 37B active · 61 layers

DeepSeek-R1

3 · 8.0B–671B

DeepSeek-R1 671B-A37B
671B · 37B active · 61 layers DeepSeek-R1-Distill-Qwen 32B
32.8B · 64 layers DeepSeek-R1-Distill-Llama 8B
8.0B · 32 layers

DeepSeek-Coder-V2

1 · 15.7B

DeepSeek-Coder-V2-Lite 16B-A2.4B
15.7B · 2.4B active · 27 layers

Gemma

Google's open models. Two things dominate the memory: a 262k vocabulary — on Gemma 3 1B the embedding table is about a third of the file — and sliding-window attention, where only every sixth layer of Gemma 3 sees the full context, and every second layer on Gemma 2. The cache grows far more slowly than the context length suggests.

6 models · 1B–27.4B · huggingface.co/google · ai.google.dev/gemma

Gemma 3

4 · 1B–27.4B

Gemma 3 27B
27.4B · 62 layers Gemma 3 12B
12.2B · 48 layers Gemma 3 4B
4.3B · 34 layers Gemma 3 1B
1B · 26 layers

Gemma 2

2 · 9.2B–27.2B

Gemma 2 27B
27.2B · 46 layers Gemma 2 9B
9.2B · 42 layers

Mistral

Dense models — 7B, NeMo 12B, Small 24B and Large 123B — alongside the two Mixtral mixture-of-experts releases and Codestral for code. Vocabulary size is not consistent across the family: 32k on 7B, Large and Codestral, 131k on NeMo and Small, which changes how much of a small model is embedding table.

7 models · 7.3B–140.6B · huggingface.co/mistralai · mistral.ai

Mistral

4 · 7.3B–122.6B

Mistral Large 2 123B
122.6B · 88 layers Mistral Small 24B
23.6B · 40 layers Mistral NeMo 12B
12.3B · 40 layers Mistral 7B v0.3
7.3B · 32 layers

Mixtral

2 · 46.7B–140.6B

Mixtral 8x22B
140.6B · 39B active · 56 layers Mixtral 8x7B
46.7B · 12.9B active · 32 layers

Codestral

1 · 22.2B

Codestral 22B
22.2B · 56 layers

OpenAI

gpt-oss 20B and 120B are mixture-of-experts models that ship natively in MXFP4, so the quantisation ladder starts at the published 4-bit weights rather than at fp16. Both read very few parameters per token — 3.6B and 5.1B — which makes them unusually fast for their size.

2 models · 20.9B–116.8B · huggingface.co/openai · github.com/openai/gpt-oss

gpt-oss 120B-A5.1B
116.8B · 5.1B active · 36 layers gpt-oss 20B-A3.6B
20.9B · 3.6B active · 24 layers

Microsoft

The Phi models are trained on curated and synthetic data to punch above their parameter count. Phi-4 14B keeps a 16K context — short by current standards, but easy on the cache. Phi-4-mini goes to 128K with a 200k vocabulary.

2 models · 3.8B–14.7B · huggingface.co/microsoft

Phi-4 14B
14.7B · 40 layers Phi-4-mini 3.8B
3.8B · 32 layers

Cohere

Command R 35B uses full multi-head attention rather than grouped-query, so its KV cache costs five times as much per token as Command R+ 104B. On this family it is context length, not weights, that usually runs you out of memory.

2 models · 35.0B–103.8B · huggingface.co/CohereForAI · cohere.com

Command R+ 104B
103.8B · 64 layers Command R 35B
35.0B · 40 layers

Zhipu

GLM-4 9B is published on Hugging Face under THUDM, the Tsinghua lab that develops the series with Zhipu AI. A 9.4B dense model with a 128K context.

1 model · 9.4B · huggingface.co/THUDM · github.com/THUDM/GLM-4

GLM-4 9B
9.4B · 40 layers

01.AI

Yi-1.5 34B lands in the awkward gap above 30B: 19.5 GB of weights at Q4_K_M, so a 24 GB card takes it only with a modest context.

1 model · 34.4B · huggingface.co/01-ai · 01.ai

Yi-1.5 34B
34.4B · 60 layers

HuggingFace

SmolLM2 comes from Hugging Face's own team and targets the small end — 1.7B parameters, a 49k vocabulary and full multi-head attention. It runs on nearly anything.

1 model · 1.7B · huggingface.co/HuggingFaceTB · github.com/huggingface/smollm

SmolLM2 1.7B
1.7B · 24 layers