Apple · apple

Apple M5

Apple M5 has 32 GB of unified memory at 154 GB/s — about 22.32 GiB usable after driver and compositor overhead. 1968 of 2118 indexed models fit at 32K context with q4_0 KV. Note only 24 GB of its 32 GB is allocatable to the GPU.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
32 GB
LPDDR5X-9600
Bandwidth
154 GB/s
128-bit bus
Tensor FP16
—
dense
TDP
—
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 1688vision language 176audio tts 21image 2video 16audio asr 39embedding 26

What fits at 32K context

largest quantization that fits, per model · 1968 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
Qwen3-Next-80B-A3B-ThinkingMoEUD-IQ1_M81.3B22.61 GiB0.84 GiB23.99 GiB0.01 GiB25±37%
CallerQ5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
Dumpling-Qwen2.5-32BQ5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
OREAL-32BQ5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
Baichuan-M2-32B-abliteratedQ5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
QwQ-32B-Preview-abliterated-linear25I1-Q5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
openhands-lm-32b-v0.1I1-Q5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
Qwen2.5-Coder-32B-abliteratedI1-Q5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
INTELLECT-2Q5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
LongWriter-Zero-32BQ5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
m1-32bI1-Q5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
XMainframe-v2-Instruct-32bI1-Q5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
Qwen2.5-Coder-32B-Python-SpecialistI1-Q5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
Qwen2.5-32b-RP-InkI1-Q5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
OpenCodeReasoning-Nemotron-32B-IOIQ5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
Qwen2.5-Coder-32B-Instruct-abliteratedQ5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
OlympicCoder-32BQ5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
OpenCodeReasoning-Nemotron-32BQ5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
OpenThinker-32BQ5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
QwQ-32B-ArliAI-RpR-v4Q5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
Qwen2.5-Coder-32B-InstructQ5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
Qwen2.5-Coder-32BQ5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
QwQ-32B-abliteratedQ5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
DeepSeek-R1-Distill-Qwen-32B-hereticI1-Q5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
InnoSpark-HPC-RM-32BI1-Q5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
OpenThinker2-32BQ5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
Qwen2.5-32B-InstructQ5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
Qwen2.5-Coder-32B-Instruct-UncensoredI1-Q5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
QwQ-32B-PreviewQ5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
DeepSeek-R1-Distill-Qwen-32B-abliteratedQ5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
TinyR1-32B-PreviewQ5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
deepseek-r1-qwen-2.5-32B-ablatedQ5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
Rombos-LLM-V2.5-Qwen-32bQ5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
Qwen2.5-32B-ArliAI-RPMax-v1.3Q5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
DeepSeek-R1-Distill-Qwen-32B-Blunt-UncensoredQ5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
QwQ-32BQ5_032.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
DeepSeek-R1-Distill-Qwen-32BQ5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
Qwen2.5-VL-32B-InstructQ5_K_S33.5B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
EVA-Qwen2.5-32B-v0.2Q5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
EVA-Qwen2.5-32B-v0.1Q5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
cogito-v1-preview-qwen-32BI1-Q5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
QwQ-32B-Snowdrop-v0I1-Q5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
DeepSeek-R1-Distill-Qwen-32B-UncensoredI1-Q5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
RoguePlanet-DeepSeek-R1-Qwen-32B-RPI1-Q5_K_S32.8B21.08 GiB2.25 GiB23.98 GiB0.02 GiB5±8.3%
WizardCoder-Python-34B-V1.0I1-Q5_K_S33.7B21.64 GiB1.69 GiB23.98 GiB0.02 GiB5±8.3%
Phind-CodeLlama-34B-Python-v1I1-Q5_K_S33.7B21.64 GiB1.69 GiB23.98 GiB0.02 GiB5±8.3%
Phind-CodeLlama-34B-v2I1-Q5_K_S33.7B21.64 GiB1.69 GiB23.98 GiB0.02 GiB5±8.3%
CodeLlama-34b-instruct-hfQ5_033.7B21.64 GiB1.69 GiB23.98 GiB0.02 GiB5±8.3%
WizardLM-1.0-Uncensored-CodeLlama-34bQ5_033.7B21.64 GiB1.69 GiB23.98 GiB0.02 GiB5±8.3%
InternVL3_5-30B-A3BQ6_K30.8B23.38 GiB0.00 GiB23.98 GiB0.02 GiB5±8.3%
OpenBuddy-R1-0528-Distill-Qwen3-32B-Preview0-QATQ5_K_S32.8B21.08 GiB2.25 GiB23.97 GiB0.03 GiB5±8.3%
Qwen3-VL-32B-Instruct-ultra-uncensored-hereticI1-Q5_K_S33.4B21.08 GiB2.25 GiB23.97 GiB0.03 GiB5±8.3%
Huihui-Qwen3-VL-32B-Instruct-abliteratedI1-Q5_K_S33.4B21.08 GiB2.25 GiB23.97 GiB0.03 GiB5±8.3%
KAT-DevQ5_K_S32.8B21.08 GiB2.25 GiB23.97 GiB0.03 GiB5±8.3%
ColorGUI-32BI1-Q5_K_S33.4B21.08 GiB2.25 GiB23.97 GiB0.03 GiB5±8.3%
Qwen3-VL-32B-InstructQ5_K_S33.4B21.08 GiB2.25 GiB23.97 GiB0.03 GiB5±8.3%
Qwen3-VL-32B-ThinkingQ5_K_S33.4B21.08 GiB2.25 GiB23.97 GiB0.03 GiB5±8.3%
Qwen3-32B-UncensoredI1-Q5_K_S32.8B21.08 GiB2.25 GiB23.97 GiB0.03 GiB5±8.3%
Qwen3-32BQ5_K_S32.8B21.08 GiB2.25 GiB23.97 GiB0.03 GiB5±8.3%
Qwen3-32B-abliteratedI1-Q5_K_S32.8B21.08 GiB2.25 GiB23.97 GiB0.03 GiB5±8.3%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a Apple M5 run?
1968 of 2118 indexed open-weight models fit a Apple M5 at 32,768 context with q4_0 KV cache, the largest being Qwen3-Next-80B-A3B-Thinking at UD-IQ1_M. That covers text, vision-language, image, video and speech models.
How much usable memory does a Apple M5 actually have?
Its nameplate is 32 GB, but about 22.32 GiB is available to a model once driver and compositor overhead is accounted for, and only 24 GB of the pool can be allocated to the GPU at all.
Is a Apple M5 fast for local AI?
Its memory bandwidth is 154 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.
Apple M5 — what AI models can it run locally? — ossmodeldb