Ministral-8B-Instruct-2410
mistralai/Ministral-8B-Instruct-2410Ministral-8B-Instruct-2410 at Q4_K_M is exactly 4,911,500,096 bytes (4.57 GiB / 4.91 GB) — an effective 4.899 bits per weight, not the nominal 4. Its KV cache at 32K is 4.50 GiB, not the 4.50 GiB a flat formula predicts.
Shipped quantizations
| Quant | Size● | Exact bytes● | Effective bpw● | Tensors● | Publisher |
|---|---|---|---|---|---|
| IQ2_M | 2.76 GiB | 2,958,330,688 | 2.951 | — | bartowski |
| Q2_K | 2.97 GiB | 3,185,478,464 | 3.178 | — | bartowski |
| IQ3_XS | 3.28 GiB | 3,521,940,288 | 3.513 | — | bartowski |
| Q3_K_S | 3.41 GiB | 3,664,677,696 | 3.656 | — | bartowski |
| Q2_K_L | 3.45 GiB | 3,709,766,464 | 3.701 | — | bartowski |
| IQ3_M | 3.53 GiB | 3,791,686,464 | 3.782 | — | bartowski |
| Q3_K_M | 3.74 GiB | 4,019,227,456 | 4.009 | — | bartowski |
| Q3_K_L | 4.03 GiB | 4,326,460,224 | 4.316 | — | bartowski |
| IQ4_XS | 4.14 GiB | 4,448,226,112 | 4.437 | — | bartowski |
| Q4_0 | 4.34 GiB | 4,658,464,992 | 4.647 | — | vanta-research |
| Q4_0 | 4.35 GiB | 4,671,048,512 | 4.660 | — | bartowski |
| Q4_K_S | 4.36 GiB | 4,685,728,576 | 4.674 | — | bartowski |
| Q4_K_M | 4.57 GiB | 4,911,500,096 | 4.899 | — | bartowski |
| Q4_K_L | 4.95 GiB | 5,309,958,976 | 5.297 | — | bartowski |
| Q5_K_S | 5.21 GiB | 5,593,795,392 | 5.580 | — | bartowski |
| Q5_K_M | 5.33 GiB | 5,724,146,496 | 5.710 | — | bartowski |
| Q5_K_L | 5.64 GiB | 6,055,496,512 | 6.040 | — | bartowski |
| Q6_K | 6.14 GiB | 6,587,583,296 | 6.571 | — | bartowski |
| Q6_K_L | 6.38 GiB | 6,847,630,144 | 6.831 | — | bartowski |
| Q8_0 | 7.94 GiB | 8,529,808,192 | 8.509 | — | bartowski |
| F16 | 14.95 GiB | 16,048,097,824 | 16.009 | — | bartowski |
KV cache by context
| Context | KV cache (f16)● | Flat formula | Overstated by | Full / windowed / recurrent |
|---|---|---|---|---|
| 4,096 | 0.56 GiB | 0.56 GiB | — | 9 / 27 / 0 |
| 8,192 | 1.13 GiB | 1.13 GiB | — | 9 / 27 / 0 |
| 16,384 | 2.25 GiB | 2.25 GiB | — | 9 / 27 / 0 |
| 32,768 | 4.50 GiB | 4.50 GiB | — | 9 / 27 / 0 |
| 65,536 | 5.68 GiB | 9.00 GiB | 1.59× | 9 / 27 / 0 |
| 131,072 | 7.93 GiB | 18.00 GiB | 2.27× | 9 / 27 / 0 |
27 of 36 layers cache only a 32,768-token window rather than the full context, on a period of . Figures assume the default configuration; --swa-full disables the saving entirely.
Compare with
Will it run on your card?
Why other calculators give a different number
A parameters × bits ÷ 8 estimate puts Q4_K_M at roughly 4.20 GiB. The real file is 4.57 GiB, because a quantization is a mixture and some tensors are always kept at higher precision. The larger discrepancy is the cache: a flat formula gives 4.50 GiB at 32K context where the real figure is 4.50 GiB, because most of this model's layers cache a fixed window rather than the whole context.
Architecture
Questions people ask
- How much VRAM does Ministral-8B-Instruct-2410 need?
- Q4_K_M is exactly 4,911,500,096 bytes (4.57 GiB) in weights. Add the KV cache, which depends on your context length, plus roughly half a gigabyte of runtime overhead.
- How large is Ministral-8B-Instruct-2410's KV cache?
- 4.50 GiB at 32K context with an f16 cache, computed per layer. Quantizing the cache to q8_0 roughly halves it, which is often the difference between a context length fitting and not.
- Which quantization of Ministral-8B-Instruct-2410 should I use?
- Q4_K_M is the usual default. Pick the largest quantization that fits your card at the context you actually need — the table above gives exact sizes for every one published.