How much VRAM does v6-Finch-14B-HF need?

Q4_K_M is exactly 8,767,898,176 bytes (8.17 GiB) in weights. Add the KV cache, which depends on context length, plus roughly half a gigabyte of runtime overhead.

How large is v6-Finch-14B-HF's KV cache?

30.50 GiB at 32K context with an f16 cache, computed per layer from the model's architecture. Quantizing the cache to q8_0 roughly halves it.

RWKV · text

v6-Finch-14B-HF

RWKV/v6-Finch-14B-HF

v6-Finch-14B-HF at Q4_K_M is exactly 8,767,898,176 bytes (8.17 GiB / 8.77 GB) — an effective 4.986 bits per weight, not the nominal 4. Its KV cache at 32K is 30.50 GiB.

From the file· summed from 1 file(s)From the file· KV per layer

Parameters

14.1B

Architecture

rwkv6

61 layers

Context

—

native (config.json)

License

apache-2.0

Shipped quantizations

● exact bytes, summed from published files

Quant	Size●	Exact bytes●	Effective bpw●	Tensors●	Publisher
IQ2_M	5.10 GiB	5,474,452,032	3.113	—	bartowski
Q2_K	5.20 GiB	5,586,780,736	3.177	—	bartowski
Q2_K_L	5.45 GiB	5,848,924,736	3.326	—	bartowski
IQ3_M	6.49 GiB	6,965,264,960	3.961	—	bartowski
Q3_K_L	6.49 GiB	6,965,264,960	3.961	—	bartowski
IQ3_XS	6.49 GiB	6,965,264,960	3.961	—	bartowski
Q3_K_M	6.49 GiB	6,965,264,960	3.961	—	bartowski
Q3_K_S	6.49 GiB	6,965,264,960	3.961	—	bartowski
IQ4_XS	7.77 GiB	8,343,749,184	4.744	—	bartowski
Q4_K_S	8.17 GiB	8,767,898,176	4.986	—	bartowski
Q4_K_M	8.17 GiB	8,767,898,176	4.986	—	bartowski
Q4_0	8.17 GiB	8,767,898,176	4.986	—	bartowski
Q4_K_L	8.35 GiB	8,967,127,616	5.099	—	bartowski
Q5_K_M	9.75 GiB	10,464,494,144	5.950	—	bartowski
Q5_K_S	9.75 GiB	10,464,494,144	5.950	—	bartowski
Q5_K_L	9.90 GiB	10,630,169,152	6.045	—	bartowski
Q6_K	11.42 GiB	12,267,127,360	6.975	—	bartowski
Q6_K_L	11.55 GiB	12,397,150,784	7.049	—	bartowski
Q8_0	14.55 GiB	15,619,293,760	8.882	—	bartowski
F16	26.63 GiB	28,595,421,504	16.260	—	bartowski

KV cache by context

computed per layer

Context	KV cache (f16)●	Flat formula	Overstated by	Full / windowed / recurrent
4,096	3.81 GiB	3.81 GiB	—	61 / 0 / 0
8,192	7.63 GiB	7.63 GiB	—	61 / 0 / 0
16,384	15.25 GiB	15.25 GiB	—	61 / 0 / 0
32,768	30.50 GiB	30.50 GiB	—	61 / 0 / 0
65,536	61.00 GiB	61.00 GiB	—	61 / 0 / 0
131,072	122.00 GiB	122.00 GiB	—	61 / 0 / 0

Compare with

same modality, comparable size

vs Qwen3-Coder-30B-A3B-Instruct vs Qwen3.6-27B vs Qwen3.8-27B vs gemma-4-12B-it-qat-q4_0-unquantized vs gemma-4-E4B-it vs Qwen3-30B-A3B-Thinking-2507 All alternatives →Will it fit on my card? →

Why other calculators give a different number

A parameters × bits ÷ 8 estimate puts Q4_K_M at roughly 7.37 GiB. The real file is 8.17 GiB, because a quantization is a mixture and some tensors are always kept at higher precision.

Architecture

from config.json

Layers

Attention heads

KV heads

Head dim

Hidden size

4096

Vocab

65,536

Sliding window

none

SWA period

—

MLA

Experts

—

Experts per token

—

use_sliding_window

—

Questions people ask

How much VRAM does v6-Finch-14B-HF need?: Q4_K_M is exactly 8,767,898,176 bytes (8.17 GiB) in weights. Add the KV cache, which depends on your context length, plus roughly half a gigabyte of runtime overhead.
How large is v6-Finch-14B-HF's KV cache?: 30.50 GiB at 32K context with an f16 cache, computed per layer. Quantizing the cache to q8_0 roughly halves it, which is often the difference between a context length fitting and not.
Which quantization of v6-Finch-14B-HF should I use?: Q4_K_M is the usual default. Pick the largest quantization that fits your card at the context you actually need — the table above gives exact sizes for every one published.