Phi-3-mini-4k-instruct IQ2_XS
This file is exactly 1,153,035,872 bytes — 1.07 GiB / 1.15 GB at an effective 2.414 bits per weight. The nominal rate for IQ2_XS is lower; mixed-precision tensors make the real figure higher, always.
Get it
llama-cli -hf bartowski/Phi-3-mini-4k-instruct-v0.3-GGUF:IQ2_XS
Downloads and runs in one step, resolving the quantization by name.
hf download bartowski/Phi-3-mini-4k-instruct-v0.3-GGUF Phi-3-mini-4k-instruct-v0.3-IQ2_XS.gguf
Straight from the Hugging Face CDN — we host nothing and earn nothing from this. Verify what you received against the exact byte count above; a size mismatch is the usual cause of a file that will not load.
Fit by accelerator
| Accelerator | Memory○ | Total @4K◐ | Total @32K◐ | Fits◐ | tok/s @4K◐ |
|---|---|---|---|---|---|
| Arc A310 4GB | 4 GB | 3.38 GiB | 13.88 GiB | yes4K only | 26±30% |
| Arc A350M 4GB | 4 GB | 3.38 GiB | 13.88 GiB | yes4K only | 23±30% |
| Arc A370M 4GB | 4 GB | 3.38 GiB | 13.88 GiB | yes4K only | 23±30% |
| Arc A530M 4GB | 4 GB | 3.38 GiB | 13.88 GiB | yes4K only | 44±30% |
| Arc Pro A30M 4GB | 4 GB | 3.38 GiB | 13.88 GiB | yes4K only | 23±30% |
| Radeon Pro W6400 | 4 GB | 3.48 GiB | 13.98 GiB | yes4K only | 31±26.5% |
| Radeon RX 6400 | 4 GB | 3.48 GiB | 13.98 GiB | yes4K only | 31±26.5% |
| Radeon RX 6500 XT | 4 GB | 3.48 GiB | 13.98 GiB | yes4K only | 35±26.5% |
| RTX A400 | 4 GB | 3.58 GiB | 14.08 GiB | yes4K only | 21±22% |
| Arc A380 6GB | 6 GB | 3.38 GiB | 13.88 GiB | yes4K only | 37±30% |
| Arc Pro A40 6GB | 6 GB | 3.38 GiB | 13.88 GiB | yes4K only | 38±30% |
| Arc Pro A50 6GB | 6 GB | 3.38 GiB | 13.88 GiB | yes4K only | 38±30% |
| GeForce RTX 2060 | 6 GB | 3.38 GiB | 13.88 GiB | yes4K only | 90±12.9% |
| GeForce RTX 3050 | 6 GB | 3.38 GiB | 13.88 GiB | yes4K only | 46±12.9% |
| GeForce RTX 3060 OEM | 6 GB | 3.38 GiB | 13.88 GiB | yes4K only | 90±12.9% |
| RTX A2000 | 6 GB | 3.58 GiB | 14.08 GiB | yes4K only | 62±22% |
| Apple M1 | 8 GB | 3.13 GiB | 13.63 GiB | yes4K only | 20±8.3% |
| Apple M2 | 8 GB | 3.13 GiB | 13.63 GiB | yes4K only | 28±8.3% |
| Apple M3 | 8 GB | 3.13 GiB | 13.63 GiB | yes4K only | 28±8.3% |
| Apple M4 | 8 GB | 3.13 GiB | 13.63 GiB | yes4K only | 32±8.3% |
| Arc A530M 8GB | 8 GB | 3.38 GiB | 13.88 GiB | yes4K only | 44±30% |
| Arc A550M 8GB | 8 GB | 3.38 GiB | 13.88 GiB | yes4K only | 44±30% |
| Arc A570M 8GB | 8 GB | 3.38 GiB | 13.88 GiB | yes4K only | 44±30% |
| Arc A580 8GB | 8 GB | 3.38 GiB | 13.88 GiB | yes4K only | 91±30% |
Memory at context
| Context | Weights● | KV (f16)● | KV (q8_0)● | Working buffer◐ | Total (f16)◐ |
|---|---|---|---|---|---|
| 4,096 | 1.07 GiB | 1.50 GiB | 0.80 GiB | 0.31 GiB | 2.88 GiB |
| 8,192 | 1.07 GiB | 3.00 GiB | 1.59 GiB | 0.31 GiB | 4.38 GiB |
| 16,384 | 1.07 GiB | 6.00 GiB | 3.19 GiB | 0.31 GiB | 7.38 GiB |
| 32,768 | 1.07 GiB | 12.00 GiB | 6.38 GiB | 0.31 GiB | 13.38 GiB |
| 65,536 | 1.07 GiB | 24.00 GiB | 12.75 GiB | 0.31 GiB | 25.38 GiB |
| 131,072 | 1.07 GiB | 48.00 GiB | 25.50 GiB | 0.31 GiB | 49.38 GiB |
Quantizing the KV cache to q8_0 is a roughly 2× lever on the dominant term at long context, and it is the single most useful setting most local users never touch. Totals here exclude the allocator reserve your driver takes, which is hardware-specific — the per-accelerator table above includes it.