Apple · apple

Apple M2 Pro

Apple M2 Pro has 32 GB of unified memory at 205 GB/s — about 22.32 GiB usable after driver and compositor overhead. 1997 of 2118 indexed models fit at 8K context with q4_0 KV. Note only 24 GB of its 32 GB is allocatable to the GPU.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
32 GB
LPDDR5-6400
Bandwidth
205 GB/s
256-bit bus
Tensor FP16
—
dense
TDP
—
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 1715vision language 178audio tts 21image 2video 16audio asr 39embedding 26

What fits at 8K context

largest quantization that fits, per model · 1997 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
Qwen3-Coder-NextMoEUD-IQ2_M79.7B23.25 GiB0.21 GiB24.00 GiB0.00 GiB38±37%
ALIA-40b-fc-2606I1-Q4_K_M40.4B22.90 GiB0.42 GiB23.99 GiB0.01 GiB7±8.3%
ALIA-40b-instruct-2606I1-Q4_K_M40.4B22.90 GiB0.42 GiB23.99 GiB0.01 GiB7±8.3%
InternVL3_5-30B-A3BQ6_K30.8B23.38 GiB0.00 GiB23.98 GiB0.02 GiB7±8.3%
Aurora-Code-1MoEI1-Q6_K34.7B23.37 GiB0.04 GiB23.97 GiB0.03 GiB37±37%
OmniAtlas-Qwen3-30B-A3BI1-Q6_K31.7B23.37 GiB0.00 GiB23.96 GiB0.04 GiB7±8.3%
Qwen3-Omni-30B-A3B-CaptionerI1-Q6_K31.7B23.37 GiB0.00 GiB23.96 GiB0.04 GiB7±8.3%
GLM-4.7-Flash-hereticMoEQ6_K29.9B23.27 GiB0.12 GiB23.95 GiB0.05 GiB29±37%
Nemotron-Labs-Audex-30B-A3BQ4_K_L32.0B23.35 GiB0.00 GiB23.95 GiB0.05 GiB7±8.3%
Llama3.2-30B-A3B-II-Dark-Champion-INSTRUCT-Heretic-Abliterated-UncensoredMoEI1-Q6_K30.0B22.97 GiB0.41 GiB23.94 GiB0.06 GiB20±37%
Qwen3.5-35B-A3BMoEQ5_K_S36.0B23.33 GiB0.04 GiB23.93 GiB0.07 GiB37±37%
Qwen3.6-35B-A3BMoEQ5_K_S36.0B23.33 GiB0.04 GiB23.93 GiB0.07 GiB37±37%
Assistant_Pepe_70BIQ2_XS70.6B22.53 GiB0.70 GiB23.91 GiB0.09 GiB7±8.3%
Darwin-35B-A3B-OpusMoEQ5_K_M36.0B23.30 GiB0.04 GiB23.90 GiB0.10 GiB37±37%
grug-35b-v2MoEQ5_K_M35.1B23.30 GiB0.04 GiB23.90 GiB0.10 GiB37±37%
grug-35bMoEQ5_K_M35.1B23.30 GiB0.04 GiB23.90 GiB0.10 GiB37±37%
WorldSim-Opus-3.6-35B-A3BMoEQ5_K_M35.1B23.30 GiB0.04 GiB23.90 GiB0.10 GiB37±37%
Qwen3.6-35B-A3B-AnkoMoEQ5_K_M35.1B23.30 GiB0.04 GiB23.90 GiB0.10 GiB37±37%
KAT-Coder-V2.5-DevMoEQ5_K_M34.7B23.30 GiB0.04 GiB23.90 GiB0.10 GiB37±37%
Ornith-1.0-35BMoEQ5_K_M34.7B23.30 GiB0.04 GiB23.90 GiB0.10 GiB37±37%
Nex-N2-miniMoEQ5_K_M35.1B23.30 GiB0.04 GiB23.90 GiB0.10 GiB37±37%
Llama-4-Scout-17B-16E-Instruct-abliterated-v2MoEKV unresolvedI1-IQ1_M109B22.88 GiB0.42 GiB23.88 GiB0.12 GiB27±37%
HarmonicHarlequin_v5-20BI1-Q4_K_M33.3B18.71 GiB4.57 GiB23.87 GiB0.13 GiB7±8.3%
Laguna-S-2.1MoEIQ1_S118B23.15 GiB0.15 GiB23.87 GiB0.13 GiB34±37%
Le-Chaton-Slim-23BMoEQ8_023.3B23.07 GiB0.23 GiB23.86 GiB0.14 GiB15±37%
Maenad-70BI1-IQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
DeepSeek-R1-Distill-Llama-70B-Uncensored-v2-Unbiased-ReasonerI1-IQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
Rombos-LLM-70b-Llama-3.3I1-IQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
L3.3-Electra-R1-70bI1-IQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
L3.3-70B-Magnum-v4-SEIQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
Latxa-Llama-3.1-70B-Instruct-v2I1-IQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
Llama-3.3_70_b_uncensored_continuedI1-IQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
Llama-3.3-70B-Instruct-abliteratedI1-IQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
grok-oss-Revenant-70BI1-IQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
Llama-3.1-Nemotron-70B-Instruct-HFI1-IQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
L3.3-70B-Euryale-v2.3I1-IQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
Hermes-3-Llama-3.1-70BIQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
Hermes-4-70B-hereticI1-IQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
Llama-3.3-70B-InstructIQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
Llama-3.1-70BIQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
Anubis-70B-v1.2IQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
Hermes-4-70BIQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
Golem-70B-v1bI1-IQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
DeepSeek-R1-Distill-Llama-70B-abliteratedI1-IQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
DeepSeek-R1-Distill-Llama-70B-hereticI1-IQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
DeepSeek-R1-Distill-Llama-70BIQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
Legion-V2.1-LLaMa-70BI1-IQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
Tess-R1-Limerick-Llama-3.1-70BIQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
SEMIKONG-70BIQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
functionary-medium-v3.2KV unresolvedIQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
Llama-3.1-WhiteRabbitNeo-2-70BIQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
Infinity-Instruct-7M-Gen-Llama3_1-70BI1-IQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
New-Dawn-Llama-3-70B-32K-v1.0I1-IQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
Meta-Llama-3-70B-Instruct-abliterated-v3.5I1-IQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
Athene-70BIQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
L3.3-70B-Magnum-DiamondIQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
Meta-Llama-3-70B-InstructIQ2_M70.6B22.46 GiB0.70 GiB23.84 GiB0.16 GiB7±8.3%
Qwen-AgentWorld-35B-A3BMoEUD-Q5_K_S34.7B23.23 GiB0.04 GiB23.83 GiB0.17 GiB37±37%
Qwen3.5-40B-Claude-4.6-Opus-Deckard-Heretic-Uncensored-ThinkingI1-Q4_139.5B23.00 GiB0.21 GiB23.82 GiB0.18 GiB7±8.3%
Qwen3.5-40B-RoughHouse-Claude-4.6-Opus-Polar-Deckard-Uncensored-Heretic-ThinkingI1-Q4_139.5B23.00 GiB0.21 GiB23.82 GiB0.18 GiB7±8.3%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a Apple M2 Pro run?
1997 of 2118 indexed open-weight models fit a Apple M2 Pro at 8,192 context with q4_0 KV cache, the largest being Qwen3-Coder-Next at UD-IQ2_M. That covers text, vision-language, image, video and speech models.
How much usable memory does a Apple M2 Pro actually have?
Its nameplate is 32 GB, but about 22.32 GiB is available to a model once driver and compositor overhead is accounted for, and only 24 GB of the pool can be allocated to the GPU at all.
Is a Apple M2 Pro fast for local AI?
Its memory bandwidth is 205 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.