How much VRAM does gemma-7b need?

Q4_K_M is exactly 5,329,758,592 bytes (4.96 GiB) in weights. Add the KV cache, which depends on context length, plus roughly half a gigabyte of runtime overhead.

How large is gemma-7b's KV cache?

14.00 GiB at 32K context with an f16 cache, computed per layer from the model's architecture. Quantizing the cache to q8_0 roughly halves it.

google · text · gated

gemma-7b

google/gemma-7b

gemma-7b at Q4_K_M is exactly 5,329,758,592 bytes (4.96 GiB / 5.33 GB) — an effective 4.994 bits per weight, not the nominal 4. Its KV cache at 32K is 14.00 GiB.

From the file· summed from 1 file(s)From the file· KV from mirror (mirror:unsloth/gemma-7b)

Parameters

8.5B

Architecture

gemma

28 layers

Context

8,192

native (config.json)

License

gemma

Shipped quantizations

● exact bytes, summed from published files

Quant	Size●	Exact bytes●	Effective bpw●	Tensors●	Publisher
I1-IQ1_S	2.01 GiB	2,160,192,800	2.024	—	mradermacher
I1-IQ1_M	2.16 GiB	2,320,035,104	2.174	—	mradermacher
I1-IQ2_XXS	2.41 GiB	2,586,438,944	2.424	—	mradermacher
I1-IQ2_XS	2.62 GiB	2,810,572,064	2.634	—	mradermacher
I1-IQ2_S	2.72 GiB	2,918,903,072	2.735	—	mradermacher
I1-IQ2_M	2.92 GiB	3,132,026,144	2.935	—	mradermacher
Q2_K	3.24 GiB	3,481,446,784	3.262	—	MaziyarPanahi
I1-Q2_K	3.24 GiB	3,481,447,712	3.262	—	mradermacher
I1-IQ3_XXS	3.25 GiB	3,487,100,192	3.268	—	mradermacher
I1-IQ3_XS	3.54 GiB	3,800,739,104	3.561	—	mradermacher
Q3_K_S	3.71 GiB	3,982,403,968	3.732	—	MaziyarPanahi
I1-Q3_K_S	3.71 GiB	3,982,404,896	3.732	—	mradermacher
I1-IQ3_S	3.71 GiB	3,982,404,896	3.732	—	mradermacher
I1-IQ3_M	3.82 GiB	4,106,071,328	3.848	—	mradermacher
Q3_K_M	4.07 GiB	4,369,328,512	4.094	—	MaziyarPanahi
I1-Q3_K_M	4.07 GiB	4,369,329,440	4.094	—	mradermacher
Q3_K_L	4.39 GiB	4,709,067,136	4.412	—	MaziyarPanahi
I1-Q3_K_L	4.39 GiB	4,709,068,064	4.412	—	mradermacher
I1-IQ4_XS	4.44 GiB	4,769,623,328	4.469	—	mradermacher
I1-Q4_0	4.68 GiB	5,026,000,160	4.710	—	mradermacher
Q4_K_S	4.70 GiB	5,046,446,464	4.729	—	MaziyarPanahi
I1-Q4_K_S	4.70 GiB	5,046,447,392	4.729	—	mradermacher
Q4_K_M	4.96 GiB	5,329,758,592	4.994	—	MaziyarPanahi
I1-Q4_K_M	4.96 GiB	5,329,759,520	4.994	—	mradermacher
Q5_K_S	5.57 GiB	5,980,727,680	5.604	—	MaziyarPanahi
I1-Q5_K_S	5.57 GiB	5,980,728,608	5.604	—	mradermacher
Q5_K_M	5.72 GiB	6,144,502,144	5.758	—	MaziyarPanahi
I1-Q5_K_M	5.72 GiB	6,144,503,072	5.758	—	mradermacher
Q6_K	6.53 GiB	7,010,167,168	6.569	—	MaziyarPanahi
I1-Q6_K	6.53 GiB	7,010,168,096	6.569	—	mradermacher
Q8_0	8.45 GiB	9,077,844,352	8.506	—	MaziyarPanahi

KV cache by context

computed per layer

Context	KV cache (f16)●	Flat formula	Overstated by	Full / windowed / recurrent
4,096	1.75 GiB	1.75 GiB	—	28 / 0 / 0
8,192	3.50 GiB	3.50 GiB	—	28 / 0 / 0
16,384	7.00 GiB	7.00 GiB	—	28 / 0 / 0
32,768	14.00 GiB	14.00 GiB	—	28 / 0 / 0
65,536	28.00 GiB	28.00 GiB	—	28 / 0 / 0
131,072	56.00 GiB	56.00 GiB	—	28 / 0 / 0

Compare with

same modality, comparable size

vs gemma-4-E4B-it vs Qwen3-4B vs Qwen3-8B vs HyperCLOVAX-SEED-Text-Instruct-1.5B vs Llama-3.2-1B-Instruct vs Qwen3-VL-8B-Instruct-abliterated-v1 All alternatives →Will it fit on my card? →

Why other calculators give a different number

A parameters × bits ÷ 8 estimate puts Q4_K_M at roughly 4.47 GiB. The real file is 4.96 GiB, because a quantization is a mixture and some tensors are always kept at higher precision.

Architecture

from mirror:unsloth/gemma-7b

Layers

Attention heads

KV heads

Head dim

256

Hidden size

3072

Vocab

256,000

Sliding window

none

SWA period

—

MLA

Experts

—

Experts per token

—

use_sliding_window

—

Questions people ask

How much VRAM does gemma-7b need?: Q4_K_M is exactly 5,329,758,592 bytes (4.96 GiB) in weights. Add the KV cache, which depends on your context length, plus roughly half a gigabyte of runtime overhead.
How large is gemma-7b's KV cache?: 14.00 GiB at 32K context with an f16 cache, computed per layer. Quantizing the cache to q8_0 roughly halves it, which is often the difference between a context length fitting and not.
Which quantization of gemma-7b should I use?: Q4_K_M is the usual default. Pick the largest quantization that fits your card at the context you actually need — the table above gives exact sizes for every one published.