Apple · apple

Apple M4

Apple M4 has 32 GB of unified memory at 120 GB/s — about 22.32 GiB usable after driver and compositor overhead. 1985 of 2118 indexed models fit at 16K context with q4_0 KV. Note only 24 GB of its 32 GB is allocatable to the GPU.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
32 GB
LPDDR5X-7500
Bandwidth
120 GB/s
128-bit bus
Tensor FP16
—
dense
TDP
—
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 1705vision language 176audio tts 21image 2video 16audio asr 39embedding 26

What fits at 16K context

largest quantization that fits, per model · 1985 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
Hunyuan-A13B-InstructMoEUD-IQ1_S80.4B22.89 GiB0.56 GiB24.00 GiB0.00 GiB4±8.3%
Hy-MT2-30B-A3BMoEQ6_K30.1B23.01 GiB0.42 GiB23.98 GiB0.02 GiB17±37%
InternVL3_5-30B-A3BQ6_K30.8B23.38 GiB0.00 GiB23.98 GiB0.02 GiB4±8.3%
Qwen3.5-35B-A3BMoEQ5_K_S36.0B23.33 GiB0.09 GiB23.97 GiB0.03 GiB23±37%
Qwen3.6-35B-A3BMoEQ5_K_S36.0B23.33 GiB0.09 GiB23.97 GiB0.03 GiB23±37%
Laguna-S-2.1MoEIQ1_S118B23.15 GiB0.25 GiB23.97 GiB0.03 GiB21±37%
EuroLLM-22B-Instruct-2512Q8_022.6B22.41 GiB0.95 GiB23.97 GiB0.03 GiB4±8.3%
OmniAtlas-Qwen3-30B-A3BI1-Q6_K31.7B23.37 GiB0.00 GiB23.96 GiB0.04 GiB4±8.3%
Qwen3-Omni-30B-A3B-CaptionerI1-Q6_K31.7B23.37 GiB0.00 GiB23.96 GiB0.04 GiB4±8.3%
Nemotron-Labs-Audex-30B-A3BQ4_K_L32.0B23.35 GiB0.00 GiB23.95 GiB0.05 GiB4±8.3%
Darwin-35B-A3B-OpusMoEQ5_K_M36.0B23.30 GiB0.09 GiB23.94 GiB0.06 GiB23±37%
Aurora-Code-1MoEQ5_K_M34.7B23.30 GiB0.09 GiB23.94 GiB0.06 GiB23±37%
grug-35b-v2MoEQ5_K_M35.1B23.30 GiB0.09 GiB23.94 GiB0.06 GiB23±37%
grug-35bMoEQ5_K_M35.1B23.30 GiB0.09 GiB23.94 GiB0.06 GiB23±37%
WorldSim-Opus-3.6-35B-A3BMoEQ5_K_M35.1B23.30 GiB0.09 GiB23.94 GiB0.06 GiB23±37%
Qwen3.6-35B-A3B-AnkoMoEQ5_K_M35.1B23.30 GiB0.09 GiB23.94 GiB0.06 GiB23±37%
KAT-Coder-V2.5-DevMoEQ5_K_M34.7B23.30 GiB0.09 GiB23.94 GiB0.06 GiB23±37%
Ornith-1.0-35BMoEQ5_K_M34.7B23.30 GiB0.09 GiB23.94 GiB0.06 GiB23±37%
Nex-N2-miniMoEQ5_K_M35.1B23.30 GiB0.09 GiB23.94 GiB0.06 GiB23±37%
solar-pro-preview-instructKV unresolvedQ8_022.1B21.91 GiB1.41 GiB23.93 GiB0.07 GiB4±8.3%
HarmonicHarlequin_v5-20BI1-IQ3_M33.3B14.18 GiB9.14 GiB23.92 GiB0.08 GiB4±8.3%
Nemotron-Cascade-2-30B-A3BMoEQ4_K_L31.6B23.15 GiB0.23 GiB23.91 GiB0.09 GiB20±37%
CallerQ5_K_L32.8B22.11 GiB1.13 GiB23.89 GiB0.11 GiB4±8.3%
Dumpling-Qwen2.5-32BQ5_K_L32.8B22.11 GiB1.13 GiB23.89 GiB0.11 GiB4±8.3%
OREAL-32BQ5_K_L32.8B22.11 GiB1.13 GiB23.89 GiB0.11 GiB4±8.3%
openhands-lm-32b-v0.1Q5_K_L32.8B22.11 GiB1.13 GiB23.89 GiB0.11 GiB4±8.3%
LongWriter-Zero-32BQ5_K_L32.8B22.11 GiB1.13 GiB23.89 GiB0.11 GiB4±8.3%
OpenCodeReasoning-Nemotron-32B-IOIQ5_K_L32.8B22.11 GiB1.13 GiB23.89 GiB0.11 GiB4±8.3%
Qwen2.5-Coder-32B-Instruct-abliteratedQ5_K_L32.8B22.11 GiB1.13 GiB23.89 GiB0.11 GiB4±8.3%
OlympicCoder-32BQ5_K_L32.8B22.11 GiB1.13 GiB23.89 GiB0.11 GiB4±8.3%
OpenCodeReasoning-Nemotron-32BQ5_K_L32.8B22.11 GiB1.13 GiB23.89 GiB0.11 GiB4±8.3%
OpenThinker-32BQ5_K_L32.8B22.11 GiB1.13 GiB23.89 GiB0.11 GiB4±8.3%
QwQ-32B-ArliAI-RpR-v4Q5_K_L32.8B22.11 GiB1.13 GiB23.89 GiB0.11 GiB4±8.3%
Qwen2.5-Coder-32B-InstructQ5_K_L32.8B22.11 GiB1.13 GiB23.89 GiB0.11 GiB4±8.3%
QwQ-32B-abliteratedQ5_K_L32.8B22.11 GiB1.13 GiB23.89 GiB0.11 GiB4±8.3%
OpenThinker2-32BQ5_K_L32.8B22.11 GiB1.13 GiB23.89 GiB0.11 GiB4±8.3%
INTELLECT-2Q5_K_L32.8B22.11 GiB1.13 GiB23.89 GiB0.11 GiB4±8.3%
Qwen2.5-32B-InstructQ5_K_L32.8B22.11 GiB1.13 GiB23.89 GiB0.11 GiB4±8.3%
QwQ-32B-PreviewQ5_K_L32.8B22.11 GiB1.13 GiB23.89 GiB0.11 GiB4±8.3%
Qwen2.5-Coder-32BQ5_K_L32.8B22.11 GiB1.13 GiB23.89 GiB0.11 GiB4±8.3%
Qwen2.5-32b-RP-InkQ5_K_L32.8B22.11 GiB1.13 GiB23.89 GiB0.11 GiB4±8.3%
TinyR1-32B-PreviewQ5_K_L32.8B22.11 GiB1.13 GiB23.89 GiB0.11 GiB4±8.3%
deepseek-r1-qwen-2.5-32B-ablatedQ5_K_L32.8B22.11 GiB1.13 GiB23.89 GiB0.11 GiB4±8.3%
Rombos-LLM-V2.5-Qwen-32bQ5_K_L32.8B22.11 GiB1.13 GiB23.89 GiB0.11 GiB4±8.3%
DeepSeek-R1-Distill-Qwen-32B-abliteratedQ5_K_L32.8B22.11 GiB1.13 GiB23.89 GiB0.11 GiB4±8.3%
Qwen2.5-32B-ArliAI-RPMax-v1.3Q5_K_L32.8B22.11 GiB1.13 GiB23.89 GiB0.11 GiB4±8.3%
DeepSeek-R1-Distill-Qwen-32BQ5_K_L32.8B22.11 GiB1.13 GiB23.89 GiB0.11 GiB4±8.3%
Qwen2.5-VL-32B-InstructQ5_K_L33.5B22.11 GiB1.13 GiB23.89 GiB0.11 GiB4±8.3%
EVA-Qwen2.5-32B-v0.2Q5_K_L32.8B22.11 GiB1.13 GiB23.89 GiB0.11 GiB4±8.3%
EVA-Qwen2.5-32B-v0.1Q5_K_L32.8B22.11 GiB1.13 GiB23.89 GiB0.11 GiB4±8.3%
cogito-v1-preview-qwen-32BQ5_K_L32.8B22.11 GiB1.13 GiB23.88 GiB0.12 GiB4±8.3%
QwQ-32B-Snowdrop-v0Q5_K_L32.8B22.11 GiB1.13 GiB23.88 GiB0.12 GiB4±8.3%
Qwen-AgentWorld-35B-A3BMoEUD-Q5_K_S34.7B23.23 GiB0.09 GiB23.87 GiB0.13 GiB23±37%
deepseek-llm-67b-chatI1-IQ2_M67.4B21.53 GiB1.67 GiB23.85 GiB0.15 GiB4±8.3%
deepseek-llm-67b-baseI1-IQ2_M67.4B21.53 GiB1.67 GiB23.85 GiB0.15 GiB4±8.3%
openbuddy-deepseek-67b-v15.3-4kI1-IQ2_M67.4B21.53 GiB1.67 GiB23.85 GiB0.15 GiB4±8.3%
spoomplesmaxx-v2.1-30BI1-Q6_K28.9B22.06 GiB1.13 GiB23.84 GiB0.16 GiB4±8.3%
Huihui-granite-4.1-30b-abliteratedI1-Q6_K28.9B22.06 GiB1.13 GiB23.84 GiB0.16 GiB4±8.3%
granite-4.1-30b-hereticI1-Q6_K28.9B22.06 GiB1.13 GiB23.84 GiB0.16 GiB4±8.3%
granite-4.1-30bQ6_K28.9B22.06 GiB1.13 GiB23.84 GiB0.16 GiB4±8.3%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a Apple M4 run?
1985 of 2118 indexed open-weight models fit a Apple M4 at 16,384 context with q4_0 KV cache, the largest being Hunyuan-A13B-Instruct at UD-IQ1_S. That covers text, vision-language, image, video and speech models.
How much usable memory does a Apple M4 actually have?
Its nameplate is 32 GB, but about 22.32 GiB is available to a model once driver and compositor overhead is accounted for, and only 24 GB of the pool can be allocated to the GPU at all.
Is a Apple M4 fast for local AI?
Its memory bandwidth is 120 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.