magnum-v4-9b
anthracite-org/magnum-v4-9bmagnum-v4-9b at Q4_K_M is exactly 5,761,057,696 bytes (5.37 GiB / 5.76 GB) — an effective 4.987 bits per weight, not the nominal 4. Its KV cache at 32K is 5.99 GiB, not the 10.50 GiB a flat formula predicts.
Shipped quantizations
| Quant | Size● | Exact bytes● | Effective bpw● | Tensors● | Publisher |
|---|---|---|---|---|---|
| IQ2_M | 3.20 GiB | 3,434,668,960 | 2.973 | — | bartowski |
| Q2_K | 3.54 GiB | 3,805,397,920 | 3.294 | — | bartowski |
| Q2_K_L | 3.75 GiB | 4,027,605,920 | 3.486 | — | bartowski |
| IQ3_XS | 3.86 GiB | 4,144,989,088 | 3.588 | — | bartowski |
| Q3_K_S | 4.04 GiB | 4,337,664,928 | 3.755 | — | bartowski |
| IQ3_M | 4.19 GiB | 4,494,615,456 | 3.891 | — | bartowski |
| Q3_K_M | 4.43 GiB | 4,761,781,152 | 4.122 | — | bartowski |
| Q3_K_L | 4.78 GiB | 5,132,452,768 | 4.443 | — | bartowski |
| IQ4_XS | 4.83 GiB | 5,183,030,176 | 4.487 | — | bartowski |
| Q4_0 | 5.08 GiB | 5,459,198,880 | 4.726 | — | bartowski |
| Q4_K_S | 5.10 GiB | 5,478,925,216 | 4.743 | — | bartowski |
| Q4_K_M | 5.37 GiB | 5,761,057,696 | 4.987 | — | bartowski |
| Q4_K_L | 5.57 GiB | 5,983,265,696 | 5.179 | — | bartowski |
| Q5_K_S | 6.04 GiB | 6,483,592,096 | 5.612 | — | bartowski |
| Q5_K_M | 6.19 GiB | 6,647,366,560 | 5.754 | — | bartowski |
| Q5_K_L | 6.40 GiB | 6,869,574,560 | 5.947 | — | bartowski |
| Q6_K | 7.07 GiB | 7,589,069,728 | 6.569 | — | bartowski |
| Q6_K_L | 7.27 GiB | 7,811,277,728 | 6.762 | — | bartowski |
| Q8_0 | 9.15 GiB | 9,827,148,704 | 8.507 | — | bartowski |
| F16 | 17.22 GiB | 18,490,679,968 | 16.006 | — | bartowski |
KV cache by context
| Context | KV cache (f16)● | Flat formula | Overstated by | Full / windowed / recurrent |
|---|---|---|---|---|
| 4,096 | 1.31 GiB | 1.31 GiB | — | 21 / 21 / 0 |
| 8,192 | 2.05 GiB | 2.63 GiB | 1.28× | 21 / 21 / 0 |
| 16,384 | 3.36 GiB | 5.25 GiB | 1.56× | 21 / 21 / 0 |
| 32,768 | 5.99 GiB | 10.50 GiB | 1.75× | 21 / 21 / 0 |
| 65,536 | 11.24 GiB | 21.00 GiB | 1.87× | 21 / 21 / 0 |
| 131,072 | 21.74 GiB | 42.00 GiB | 1.93× | 21 / 21 / 0 |
21 of 42 layers cache only a 4,096-token window rather than the full context, on a period of 2. Figures assume the default configuration; --swa-full disables the saving entirely.
Compare with
Will it run on your card?
Why other calculators give a different number
A parameters × bits ÷ 8 estimate puts Q4_K_M at roughly 4.84 GiB. The real file is 5.37 GiB, because a quantization is a mixture and some tensors are always kept at higher precision. The larger discrepancy is the cache: a flat formula gives 10.50 GiB at 32K context where the real figure is 5.99 GiB, because most of this model's layers cache a fixed window rather than the whole context.
Architecture
Questions people ask
- How much VRAM does magnum-v4-9b need?
- Q4_K_M is exactly 5,761,057,696 bytes (5.37 GiB) in weights. Add the KV cache, which depends on your context length, plus roughly half a gigabyte of runtime overhead.
- How large is magnum-v4-9b's KV cache?
- 5.99 GiB at 32K context with an f16 cache, computed per layer. Quantizing the cache to q8_0 roughly halves it, which is often the difference between a context length fitting and not.
- Which quantization of magnum-v4-9b should I use?
- Q4_K_M is the usual default. Pick the largest quantization that fits your card at the context you actually need — the table above gives exact sizes for every one published.