Can I run Kimi-K2-Instruct on a Apple M5 Max?
Not at these settings. No indexed quantization of Kimi-K2-Instruct fits Apple M5 Max at any context we compute, with q8_0 KV. The smallest shipped quantization is 226.87 GiB in weights alone, against 25.11 GiB usable. CPU offload can still run it, slowly.
Every quantization at every context
| Quant | Weights● | 4K◐ | 8K◐ | 16K◐ | 32K◐ | 64K◐ | 128K◐ |
|---|---|---|---|---|---|---|---|
| BF16 | 1912.15 GiB | 1912.9 | 1913.1 | 1913.3 | 1913.9 | 1915.1 | 1917.3 |
| Q8_0 | 1016.12 GiB | 1016.9 | 1017.0 | 1017.3 | 1017.9 | 1019.0 | 1021.3 |
| Q6_K | 784.76 GiB | 785.5 | 785.7 | 786.0 | 786.5 | 787.7 | 790.0 |
| Q5_K_M | 678.33 GiB | 679.1 | 679.2 | 679.5 | 680.1 | 681.2 | 683.5 |
| Q5_K_S | 658.04 GiB | 658.8 | 659.0 | 659.2 | 659.8 | 660.9 | 663.2 |
| Q4_1 | 598.40 GiB | 599.2 | 599.3 | 599.6 | 600.2 | 601.3 | 603.6 |
| Q4_K_M | 578.15 GiB | 578.9 | 579.1 | 579.3 | 579.9 | 581.1 | 583.3 |
| Q4_K_S | 542.73 GiB | 543.5 | 543.6 | 543.9 | 544.5 | 545.6 | 547.9 |
| Q4_0 | 540.74 GiB | 541.5 | 541.7 | 541.9 | 542.5 | 543.7 | 545.9 |
| IQ4_NL | 538.76 GiB | 539.5 | 539.7 | 540.0 | 540.5 | 541.7 | 543.9 |
| IQ4_XS | 508.98 GiB | 509.8 | 509.9 | 510.2 | 510.7 | 511.9 | 514.2 |
| Q3_K_M | 455.77 GiB | 456.5 | 456.7 | 457.0 | 457.5 | 458.7 | 461.0 |
| Q3_K_S | 412.03 GiB | 412.8 | 412.9 | 413.2 | 413.8 | 414.9 | 417.2 |
| UD-IQ3_XXS | 388.01 GiB | 388.8 | 388.9 | 389.2 | 389.8 | 390.9 | 393.2 |
| Q2_K_L | 347.81 GiB | 348.6 | 348.7 | 349.0 | 349.6 | 350.7 | 353.0 |
| Q2_K | 347.55 GiB | 348.3 | 348.5 | 348.7 | 349.3 | 350.5 | 352.7 |
| UD-IQ2_M | 323.27 GiB | 324.0 | 324.2 | 324.5 | 325.0 | 326.2 | 328.5 |
| UD-IQ2_XXS | 306.20 GiB | 307.0 | 307.1 | 307.4 | 308.0 | 309.1 | 311.4 |
| UD-IQ1_M | 283.34 GiB | 284.1 | 284.3 | 284.5 | 285.1 | 286.2 | 288.5 |
| UD-IQ1_S | 260.88 GiB | 261.6 | 261.8 | 262.1 | 262.6 | 263.8 | 266.1 |
| UD-TQ1_0 | 226.87 GiB | 227.6 | 227.8 | 228.1 | 228.6 | 229.8 | 232.1 |
Figures are GiB of total memory: weights plus KV cache plus compute buffer and backend overhead. Weights and KV are near-exact; the overhead term is modeled. Hover any cell for the breakdown.
Why other calculators disagree
A parameters × bits ÷ 8 estimate ignores two things that dominate at long context. First, the weights themselves are not the nominal rate — quantizations are mixtures, so the real file is consistently larger than the label implies. Second, this model uses latent attention and allocates no V cache at all, so any formula reading num_key_value_heads overstates its cache by more than an order of magnitude.