How much VRAM does Nemotron-3-Embed-8B-BF16 need?

Q4_K_M is exactly 4,896,389,984 bytes (4.56 GiB) in weights. Add the KV cache, which depends on context length, plus roughly half a gigabyte of runtime overhead.

nvidia · embedding

Nemotron-3-Embed-8B-BF16

nvidia/Nemotron-3-Embed-8B-BF16

Nemotron-3-Embed-8B-BF16 at Q4_K_M is exactly 4,896,389,984 bytes (4.56 GiB / 4.90 GB) — an effective 4.926 bits per weight, not the nominal 4.

From the file· summed from 1 file(s)From the file· KV per layer

Parameters

8.0B

Architecture

mistral3

34 layers

Context

262,144

native (config.json)

License

openmdw-1.1

Shipped quantizations

● exact bytes, summed from published files

Quant	Size●	Exact bytes●	Effective bpw●	Tensors●	Publisher
IQ1_S	1.81 GiB	1,945,664,352	1.957	—	Aqua00
IQ1_M	1.95 GiB	2,097,249,120	2.110	—	Aqua00
IQ2_XXS	2.19 GiB	2,349,890,400	2.364	—	Aqua00
Q2_0	2.36 GiB	2,535,029,600	2.550	—	Aqua00
IQ2_XS	2.39 GiB	2,569,829,216	2.585	—	Aqua00
IQ2_S	2.49 GiB	2,673,900,384	2.690	—	Aqua00
IQ2_M	2.68 GiB	2,876,013,408	2.893	—	Aqua00
Q2_K_S	2.77 GiB	2,971,106,144	2.989	—	Aqua00
Q2_K	2.96 GiB	3,176,758,112	3.196	—	Aqua00
IQ3_XXS	3.00 GiB	3,224,664,928	3.244	—	Aqua00
IQ3_XS	3.24 GiB	3,483,663,200	3.504	—	Aqua00
Q3_K_S	3.39 GiB	3,635,772,256	3.657	—	Aqua00
IQ3_S	3.40 GiB	3,654,712,160	3.676	—	Aqua00
IQ3_M	3.50 GiB	3,761,666,912	3.784	—	Aqua00
Q3_K_M	3.74 GiB	4,011,359,072	4.035	—	Aqua00
Q3_K_M	3.74 GiB	4,011,359,127	4.035	—	Abiray
Q3_K_L	4.04 GiB	4,334,320,480	4.360	—	Aqua00
IQ4_XS	4.11 GiB	4,411,194,208	4.437	—	Aqua00
Q4_0	4.32 GiB	4,635,327,328	4.663	—	Aqua00
IQ4_NL	4.32 GiB	4,638,473,056	4.666	—	Aqua00
Q4_K_S	4.33 GiB	4,652,104,544	4.680	—	Aqua00
Q4_K_S	4.33 GiB	4,652,104,599	4.680	—	Abiray
Q4_K_M	4.56 GiB	4,896,389,984	4.926	—	Aqua00
Q4_K_M	4.56 GiB	4,896,390,039	4.926	—	Abiray
Q4_1	4.73 GiB	5,084,117,856	5.114	—	Aqua00
Q5_K_S	5.17 GiB	5,547,588,448	5.581	—	Aqua00
Q5_K_S	5.17 GiB	5,547,588,503	5.581	—	Abiray
Q5_0	5.18 GiB	5,562,268,512	5.595	—	Aqua00
Q5_K_M	5.30 GiB	5,689,637,728	5.723	—	Aqua00
Q5_K_M	5.30 GiB	5,689,637,783	5.723	—	Abiray
Q5_1	5.60 GiB	6,011,059,040	6.047	—	Aqua00
Q6_K	6.08 GiB	6,532,463,456	6.571	—	Aqua00
Q6_K	6.08 GiB	6,532,463,509	6.571	—	Abiray
Q8_0	7.88 GiB	8,458,435,424	8.509	—	Aqua00
Q8_0	7.88 GiB	8,458,435,477	8.509	—	Abiray
BF16	14.82 GiB	15,913,810,496	16.009	—	Aqua00

No KV cache

architectural, not a gap in our data

This architecture allocates no KV cache. Encoder and embedding models process their input in one pass rather than generating token by token, so there is nothing to carry forward between steps and memory does not grow with context. Its footprint is the weights plus a working buffer, and that is the whole story.

Compare with

same modality, comparable size

vs jina-embeddings-v5-text-small vs nomic-embed-text-v2-moe vs snowflake-arctic-embed-l-v2.0 vs Qwen3.5-9B-DFlash vs Qwen3-Embedding-0.6B vs Qwen3-Embedding-8B All alternatives →Will it fit on my card? →

Why other calculators give a different number

A parameters × bits ÷ 8 estimate puts Q4_K_M at roughly 4.17 GiB. The real file is 4.56 GiB, because a quantization is a mixture and some tensors are always kept at higher precision.

Architecture

from config.json

Layers

Attention heads

KV heads

Head dim

128

Hidden size

4096

Vocab

131,072

Sliding window

none

SWA period

—

MLA

Experts

—

Experts per token

—

use_sliding_window

—

Questions people ask

How much VRAM does Nemotron-3-Embed-8B-BF16 need?: Q4_K_M is exactly 4,896,389,984 bytes (4.56 GiB) in weights. Add the KV cache, which depends on your context length, plus roughly half a gigabyte of runtime overhead.
Which quantization of Nemotron-3-Embed-8B-BF16 should I use?: Q4_K_M is the usual default. Pick the largest quantization that fits your card at the context you actually need — the table above gives exact sizes for every one published.