How much VRAM does gpt2 need?

Q4_K_M is exactly 112,858,624 bytes (0.11 GiB) in weights. Add the KV cache, which depends on context length, plus roughly half a gigabyte of runtime overhead.

openai-community · text

gpt2

openai-community/gpt2

gpt2 at Q4_K_M is exactly 112,858,624 bytes (0.11 GiB / 0.11 GB) — an effective 6.589 bits per weight, not the nominal 4.

From the file· summed from 1 file(s)From the file· KV per layer

Parameters

137M

Architecture

gpt2

12 layers

Context

—

native (config.json)

License

—

Shipped quantizations

● exact bytes, summed from published files

Quant	Size●	Exact bytes●	Effective bpw●	Tensors●	Publisher
I1-IQ1_S	0.06 GiB	61,952,064	3.617	—	mradermacher
I1-IQ1_M	0.06 GiB	63,721,536	3.720	—	mradermacher
I1-IQ2_XXS	0.06 GiB	66,670,656	3.893	—	mradermacher
I1-IQ2_XS	0.06 GiB	69,251,136	4.043	—	mradermacher
I1-IQ2_S	0.07 GiB	74,406,144	4.344	—	mradermacher
I1-IQ2_M	0.07 GiB	76,765,440	4.482	—	mradermacher
Q2_K	0.08 GiB	81,196,288	4.741	—	QuantFactory
Q2_K	0.08 GiB	81,196,512	4.741	—	PrunaAI
I1-Q2_K	0.08 GiB	81,197,184	4.741	—	mradermacher
I1-IQ3_XXS	0.08 GiB	81,373,440	4.751	—	mradermacher
I1-IQ3_XS	0.08 GiB	89,153,856	5.205	—	mradermacher
Q3_K_S	0.08 GiB	90,148,288	5.263	—	QuantFactory
Q3_K_S	0.08 GiB	90,148,512	5.263	—	PrunaAI
I1-IQ3_S	0.08 GiB	90,149,184	5.263	—	mradermacher
I1-Q3_K_S	0.08 GiB	90,149,184	5.263	—	mradermacher
I1-IQ3_M	0.09 GiB	94,222,656	5.501	—	mradermacher
Q3_K_M	0.09 GiB	97,668,544	5.702	—	QuantFactory
Q3_K_M	0.09 GiB	97,668,768	5.702	—	PrunaAI
I1-Q3_K_M	0.09 GiB	97,669,440	5.702	—	mradermacher
Q3_K_L	0.10 GiB	102,092,224	5.961	—	QuantFactory
Q3_K_L	0.10 GiB	102,092,448	5.961	—	PrunaAI
I1-Q3_K_L	0.10 GiB	102,093,120	5.961	—	mradermacher
I1-IQ4_XS	0.10 GiB	102,695,392	5.996	—	mradermacher
Q4_0	0.10 GiB	106,554,880	6.221	—	QuantFactory
Q4_0	0.10 GiB	106,555,104	6.221	—	PrunaAI
I1-Q4_0	0.10 GiB	106,703,232	6.230	—	mradermacher
Q4_K_S	0.10 GiB	106,849,792	6.238	—	QuantFactory
Q4_K_S	0.10 GiB	106,850,016	6.238	—	PrunaAI
I1-Q4_K_S	0.10 GiB	106,850,688	6.238	—	mradermacher
Q4_K_M	0.11 GiB	112,858,624	6.589	—	QuantFactory
Q4_K_M	0.11 GiB	112,858,848	6.589	—	PrunaAI
I1-Q4_K_M	0.11 GiB	112,859,520	6.589	—	mradermacher
Q4_1	0.11 GiB	114,275,616	6.672	—	QuantFactory
Q4_1	0.11 GiB	114,275,840	6.672	—	PrunaAI
Q5_K_S	0.11 GiB	121,996,384	7.123	—	QuantFactory
Q5_0	0.11 GiB	121,996,384	7.123	—	QuantFactory
Q5_K_S	0.11 GiB	121,996,608	7.123	—	PrunaAI
Q5_0	0.11 GiB	121,996,608	7.123	—	PrunaAI
I1-Q5_K_S	0.11 GiB	121,997,280	7.123	—	mradermacher
Q5_K_M	0.12 GiB	126,696,544	7.397	—	QuantFactory

Compare with

same modality, comparable size

vs ced-base vs SmolLM2-135M-Instruct vs tinygemma3_cifar vs LFM2.5-230M vs TinyMistral-248M-v2-Instruct vs Qwen3.6-35B-A3B-DFlash All alternatives →Will it fit on my card? →

Why other calculators give a different number

A parameters × bits ÷ 8 estimate puts Q4_K_M at roughly 0.07 GiB. The real file is 0.11 GiB, because a quantization is a mixture and some tensors are always kept at higher precision.

Architecture

from config.json

Layers

Attention heads

—

KV heads

—

Head dim

—

Hidden size

—

Vocab

50,257

Sliding window

none

SWA period

—

MLA

Experts

—

Experts per token

—

use_sliding_window

—

Questions people ask

How much VRAM does gpt2 need?: Q4_K_M is exactly 112,858,624 bytes (0.11 GiB) in weights. Add the KV cache, which depends on your context length, plus roughly half a gigabyte of runtime overhead.
Which quantization of gpt2 should I use?: Q4_K_M is the usual default. Pick the largest quantization that fits your card at the context you actually need — the table above gives exact sizes for every one published.