How much VRAM does Llama-3.2-3B-Instruct need?

Q4_K_M is exactly 2,019,377,440 bytes (1.88 GiB) in weights. Add the KV cache, which depends on context length, plus roughly half a gigabyte of runtime overhead.

How large is Llama-3.2-3B-Instruct's KV cache?

3.50 GiB at 32K context with an f16 cache, computed per layer from the model's architecture. Quantizing the cache to q8_0 roughly halves it.

meta-llama · text

Llama-3.2-3B-Instruct

meta-llama/Llama-3.2-3B-Instruct

Llama-3.2-3B-Instruct at Q4_K_M is exactly 2,019,377,440 bytes (1.88 GiB / 2.02 GB) — an effective 5.028 bits per weight, not the nominal 4. Its KV cache at 32K is 3.50 GiB.

From the file· summed from 1 file(s)From the file· KV from mirror (mirror:unsloth/Llama-3.2-3B-Instruct)

Parameters

3.2B

Architecture

llama

28 layers

Context

131,072

native (config.json)

License

—

Shipped quantizations

● exact bytes, summed from published files

Quant	Size●	Exact bytes●	Effective bpw●	Tensors●	Publisher
UD-IQ1_S	0.85 GiB	912,345,536	2.272	—	unsloth
UD-IQ1_M	0.89 GiB	960,416,192	2.392	—	unsloth
UD-IQ2_XXS	0.97 GiB	1,046,579,648	2.606	—	unsloth
UD-IQ2_M	1.17 GiB	1,256,262,080	3.128	—	unsloth
Q2_K	1.27 GiB	1,363,935,680	3.396	—	unsloth
Q2_K_L	1.27 GiB	1,363,935,680	3.396	—	unsloth
UD-IQ3_XXS	1.28 GiB	1,370,147,264	3.412	—	unsloth
Q3_K_S	1.44 GiB	1,542,848,960	3.842	—	unsloth
IQ3_M	1.49 GiB	1,599,668,768	3.983	—	bartowski
Q3_K_M	1.57 GiB	1,687,159,232	4.201	255	unsloth
Q3_K_L	1.69 GiB	1,815,347,488	4.520	—	lmstudio-community
Q3_K_L	1.69 GiB	1,815,347,744	4.520	—	bartowski
IQ4_XS	1.70 GiB	1,829,110,208	4.555	255	unsloth
IQ4_XS	1.70 GiB	1,829,110,304	4.555	255	bartowski
IQ4_NL	1.79 GiB	1,917,190,592	4.774	—	unsloth
Q4_0	1.79 GiB	1,921,909,184	4.786	255	unsloth
Q4_0	1.79 GiB	1,921,909,280	4.786	—	bartowski
Q4_K_S	1.80 GiB	1,928,200,640	4.801	—	unsloth
Q4_K_S	1.80 GiB	1,928,200,736	4.801	—	bartowski
Q4_K_M	1.88 GiB	2,019,377,440	5.028	255	lmstudio-community
Q4_K_M	1.88 GiB	2,019,377,600	5.028	255	unsloth
Q4_K_M	1.88 GiB	2,019,377,696	5.028	255	bartowski
Q4_1	1.95 GiB	2,093,351,360	5.213	—	unsloth
Q4_K_L	1.97 GiB	2,114,800,160	5.266	—	bartowski
Q5_K_S	2.11 GiB	2,269,512,128	5.651	—	unsloth
Q5_K_S	2.11 GiB	2,269,512,224	5.651	—	bartowski
Q5_K_M	2.16 GiB	2,322,153,920	5.782	255	unsloth
Q5_K_M	2.16 GiB	2,322,154,016	5.782	255	bartowski
Q5_K_L	2.25 GiB	2,417,576,480	6.020	—	bartowski
Q6_K	2.46 GiB	2,643,853,600	6.583	—	lmstudio-community
Q6_K	2.46 GiB	2,643,853,760	6.583	255	unsloth
Q6_K	2.46 GiB	2,643,853,856	6.583	255	bartowski
Q6_K_L	2.55 GiB	2,739,276,320	6.821	—	bartowski
Q8_0	3.19 GiB	3,421,898,816	8.521	255	unsloth
Q8_0	3.19 GiB	3,421,899,040	8.521	255	lmstudio-community
Q8_0	3.19 GiB	3,421,899,296	8.521	—	bartowski
F16	5.99 GiB	6,433,687,616	16.020	255	unsloth
BF16	5.99 GiB	6,433,687,744	16.020	—	unsloth
F16	5.99 GiB	6,433,687,840	16.020	—	bartowski

KV cache by context

computed per layer

Context	KV cache (f16)●	Flat formula	Overstated by	Full / windowed / recurrent
4,096	0.44 GiB	0.44 GiB	—	28 / 0 / 0
8,192	0.88 GiB	0.88 GiB	—	28 / 0 / 0
16,384	1.75 GiB	1.75 GiB	—	28 / 0 / 0
32,768	3.50 GiB	3.50 GiB	—	28 / 0 / 0
65,536	7.00 GiB	7.00 GiB	—	28 / 0 / 0
131,072	14.00 GiB	14.00 GiB	—	28 / 0 / 0

Compare with

same modality, comparable size

vs Qwen3-4B vs Qwen3-8B vs HyperCLOVAX-SEED-Text-Instruct-1.5B vs Llama-3.2-1B-Instruct vs Qwen3-VL-8B-Instruct-abliterated-v1 vs Llama-3.1-8B-Instruct All alternatives →Will it fit on my card? →

Why other calculators give a different number

A parameters × bits ÷ 8 estimate puts Q4_K_M at roughly 1.68 GiB. The real file is 1.88 GiB, because a quantization is a mixture and some tensors are always kept at higher precision.

Architecture

from mirror:unsloth/Llama-3.2-3B-Instruct

Layers

Attention heads

KV heads

Head dim

128

Hidden size

3072

Vocab

128,256

Sliding window

none

SWA period

—

MLA

Experts

—

Experts per token

—

use_sliding_window

—

Questions people ask

How much VRAM does Llama-3.2-3B-Instruct need?: Q4_K_M is exactly 2,019,377,440 bytes (1.88 GiB) in weights. Add the KV cache, which depends on your context length, plus roughly half a gigabyte of runtime overhead.
How large is Llama-3.2-3B-Instruct's KV cache?: 3.50 GiB at 32K context with an f16 cache, computed per layer. Quantizing the cache to q8_0 roughly halves it, which is often the difference between a context length fitting and not.
Which quantization of Llama-3.2-3B-Instruct should I use?: Q4_K_M is the usual default. Pick the largest quantization that fits your card at the context you actually need — the table above gives exact sizes for every one published.