How much VRAM does Meta-Llama-3-70B-Instruct need?

Q4_K_M is exactly 42,520,393,120 bytes (39.60 GiB) in weights. Add the KV cache, which depends on context length, plus roughly half a gigabyte of runtime overhead.

How large is Meta-Llama-3-70B-Instruct's KV cache?

10.00 GiB at 32K context with an f16 cache, computed per layer from the model's architecture. Quantizing the cache to q8_0 roughly halves it.

meta-llama · text

Meta-Llama-3-70B-Instruct

meta-llama/Meta-Llama-3-70B-Instruct

Meta-Llama-3-70B-Instruct at Q4_K_M is exactly 42,520,393,120 bytes (39.60 GiB / 42.52 GB) — an effective 4.821 bits per weight, not the nominal 4. Its KV cache at 32K is 10.00 GiB.

From the file· summed from 1 file(s)From the file· KV from mirror (mirror:NousResearch/Meta-Llama-3-70B-Instruct)

Parameters

70.6B

Architecture

llama

80 layers

Context

8,192

native (config.json)

License

other

Shipped quantizations

● exact bytes, summed from published files

Quant	Size●	Exact bytes●	Effective bpw●	Tensors●	Publisher
IQ1_S	14.29 GiB	15,343,482,272	1.740	—	qwp4w3hyb
IQ1_M	15.60 GiB	16,751,195,552	1.899	—	lmstudio-community
IQ2_XXS	17.79 GiB	19,097,384,352	2.165	—	qwp4w3hyb
IQ2_XS	19.69 GiB	21,142,107,552	2.397	—	lmstudio-community
IQ2_XS	19.69 GiB	21,142,107,552	2.397	—	qwp4w3hyb
IQ2_S	20.71 GiB	22,242,342,304	2.522	—	qwp4w3hyb
IQ2_M	22.46 GiB	24,119,293,344	2.735	—	qwp4w3hyb
Q2_K	24.56 GiB	26,375,108,224	2.991	—	LiteLLMs
IQ3_XXS	25.58 GiB	27,469,493,664	3.115	—	qwp4w3hyb
IQ3_XS	27.29 GiB	29,307,729,312	3.323	—	qwp4w3hyb
IQ3_S	28.79 GiB	30,912,050,592	3.505	—	qwp4w3hyb
Q3_K_S	28.79 GiB	30,912,050,816	3.505	—	LiteLLMs
IQ3_M	29.74 GiB	31,937,033,632	3.621	—	qwp4w3hyb
Q3_K_M3 shards	31.92 GiB	34,275,312,928	3.886	—	LiteLLMs
Q3_K_L3 shards	34.60 GiB	37,148,411,168	4.212	—	LiteLLMs
IQ4_XS	35.30 GiB	37,902,661,024	4.298	—	qwp4w3hyb
Q4_03 shards	37.23 GiB	39,977,551,136	4.533	—	LiteLLMs
IQ4_NL	37.30 GiB	40,053,618,080	4.542	—	qwp4w3hyb
Q4_K_S	37.58 GiB	40,347,219,360	4.575	—	qwp4w3hyb
Q4_K_S3 shards	37.58 GiB	40,355,038,496	4.576	—	LiteLLMs
Q4_K_M	39.60 GiB	42,520,393,120	4.821	—	lmstudio-community
Q4_K_M	39.60 GiB	42,520,393,120	4.821	—	qwp4w3hyb
Q4_K_M3 shards	39.61 GiB	42,528,212,256	4.822	—	LiteLLMs
Q4_13 shards	41.28 GiB	44,321,408,288	5.026	—	LiteLLMs
Q5_K_S	45.32 GiB	48,657,446,304	5.517	—	qwp4w3hyb
Q5_03 shards	45.32 GiB	48,665,265,408	5.518	—	LiteLLMs
Q5_K_S3 shards	45.32 GiB	48,665,265,408	5.518	—	LiteLLMs
Q5_K_M	46.52 GiB	49,949,816,224	5.664	—	qwp4w3hyb
Q5_K_M3 shards	46.53 GiB	49,957,635,360	5.665	—	LiteLLMs
Q5_13 shards	49.37 GiB	53,009,122,592	6.011	—	LiteLLMs
Q6_K3 shards	53.92 GiB	57,895,961,888	6.565	—	LiteLLMs
Q8_04 shards	69.83 GiB	74,982,868,352	8.502	—	LiteLLMs

KV cache by context

computed per layer

Context	KV cache (f16)●	Flat formula	Overstated by	Full / windowed / recurrent
4,096	1.25 GiB	1.25 GiB	—	80 / 0 / 0
8,192	2.50 GiB	2.50 GiB	—	80 / 0 / 0
16,384	5.00 GiB	5.00 GiB	—	80 / 0 / 0
32,768	10.00 GiB	10.00 GiB	—	80 / 0 / 0
65,536	20.00 GiB	20.00 GiB	—	80 / 0 / 0
131,072	40.00 GiB	40.00 GiB	—	80 / 0 / 0

Compare with

same modality, comparable size

vs Qwen3-Coder-30B-A3B-Instruct vs Qwen3.6-27B vs Qwen3.8-27B vs gemma-4-12B-it-qat-q4_0-unquantized vs gemma-4-E4B-it vs Qwen3-30B-A3B-Thinking-2507 All alternatives →Will it fit on my card? →

Why other calculators give a different number

A parameters × bits ÷ 8 estimate puts Q4_K_M at roughly 36.96 GiB. The real file is 39.60 GiB, because a quantization is a mixture and some tensors are always kept at higher precision.

Architecture

from mirror:NousResearch/Meta-Llama-3-70B-Instruct

Layers

Attention heads

KV heads

Head dim

128

Hidden size

8192

Vocab

128,256

Sliding window

none

SWA period

—

MLA

Experts

—

Experts per token

—

use_sliding_window

—

Questions people ask

How much VRAM does Meta-Llama-3-70B-Instruct need?: Q4_K_M is exactly 42,520,393,120 bytes (39.60 GiB) in weights. Add the KV cache, which depends on your context length, plus roughly half a gigabyte of runtime overhead.
How large is Meta-Llama-3-70B-Instruct's KV cache?: 10.00 GiB at 32K context with an f16 cache, computed per layer. Quantizing the cache to q8_0 roughly halves it, which is often the difference between a context length fitting and not.
Which quantization of Meta-Llama-3-70B-Instruct should I use?: Q4_K_M is the usual default. Pick the largest quantization that fits your card at the context you actually need — the table above gives exact sizes for every one published.