Apple · apple

Apple M4

Apple M4 has 12 GB of unified memory at 120 GB/s — about 8.37 GiB usable after driver and compositor overhead. 802 of 2118 indexed models fit at 64K context with f16 KV. Note only 9 GB of its 12 GB is allocatable to the GPU.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
12 GB
LPDDR5X-7500
Bandwidth
120 GB/s
128-bit bus
Tensor FP16
—
dense
TDP
—
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 625video 12vision language 87audio tts 20embedding 21audio asr 36image 1

What fits at 64K context

largest quantization that fits, per model · 802 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
gemma-4-12BIQ2_S12.0B3.93 GiB4.47 GiB9.00 GiB0.00 GiB11±8.3%
Wan2.1-T2V-14BQ4_014.3B8.41 GiB0.00 GiB9.00 GiB0.00 GiB11±8.3%
Dolphin3.0-Llama3.2-3BQ3_K_S3.2B1.44 GiB7.00 GiB9.00 GiB0.00 GiB11±8.3%
Llama-Doctor-3.2-3B-InstructI1-IQ3_S3.2B1.44 GiB7.00 GiB9.00 GiB0.00 GiB11±8.3%
Llama-Song-Stream-3B-InstructQ3_K_S3.2B1.44 GiB7.00 GiB9.00 GiB0.00 GiB11±8.3%
Llama-3.2-3B-Instruct-roleplay-tunedI1-IQ3_S3.2B1.44 GiB7.00 GiB9.00 GiB0.00 GiB11±8.3%
Llama-3.2-3B-Instruct-heretic-ablitered-uncensoredI1-IQ3_S3.2B1.44 GiB7.00 GiB9.00 GiB0.00 GiB11±8.3%
llama-3.2-Korean-Bllossom-3BQ3_K_S3.2B1.44 GiB7.00 GiB9.00 GiB0.00 GiB11±8.3%
llama-3.2-3b-instruct-bnb-4bitQ3_K_S3.3B1.44 GiB7.00 GiB9.00 GiB0.00 GiB11±8.3%
Llama-3.2-3BQ3_K_S3.2B1.44 GiB7.00 GiB9.00 GiB0.00 GiB11±8.3%
Llama-3.2-3B-InstructQ3_K_S3.2B1.44 GiB7.00 GiB9.00 GiB0.00 GiB11±8.3%
Llama3.2-3B-creative-writer-v0.1I1-IQ3_S3.2B1.44 GiB7.00 GiB9.00 GiB0.00 GiB11±8.3%
Firefly-V3.2I1-IQ3_S3.2B1.44 GiB7.00 GiB9.00 GiB0.00 GiB11±8.3%
Firefly-V3I1-IQ3_S3.2B1.44 GiB7.00 GiB9.00 GiB0.00 GiB11±8.3%
Hermes-3-Llama-3.2-3BQ3_K_S3.2B1.44 GiB7.00 GiB9.00 GiB0.00 GiB11±8.3%
OpenClaude-1.7B-MergedQ6_K_L1.7B1.45 GiB7.00 GiB8.99 GiB0.01 GiB11±8.3%
Gemma-4-12B-StyleTuneI1-IQ2_XS13.0B3.93 GiB4.47 GiB8.99 GiB0.01 GiB11±8.3%
gemma-4-12b-heretic-styletune-headI1-IQ2_XS12.0B3.93 GiB4.47 GiB8.99 GiB0.01 GiB11±8.3%
syrian-gemma-12bI1-IQ2_XS13.0B3.93 GiB4.47 GiB8.99 GiB0.01 GiB11±8.3%
Gemma-4-E4B-it-Minecraft-MT-en-zh-v0.1Q8_08.0B7.48 GiB0.94 GiB8.99 GiB0.01 GiB11±8.3%
supergemma4-e4b-abliteratedQ8_07.5B7.48 GiB0.94 GiB8.99 GiB0.01 GiB11±8.3%
Gemma-4-E4B-LuchadorQ8_08.0B7.48 GiB0.94 GiB8.99 GiB0.01 GiB11±8.3%
Gemma-4-E4B-AbliteratedQ8_08.0B7.48 GiB0.94 GiB8.99 GiB0.01 GiB11±8.3%
gemma-4-E4B-it-ultra-uncensored-hereticQ8_08.0B7.48 GiB0.94 GiB8.99 GiB0.01 GiB11±8.3%
gemma-4-E4B-it-The-DECKARD-Claude-Opus-Expresso-Universe-HERETIC-UNCENSORED-ThinkingQ8_08.0B7.48 GiB0.94 GiB8.99 GiB0.01 GiB11±8.3%
Huihui-gemma-4-E4B-it-abliteratedQ8_08.0B7.48 GiB0.94 GiB8.99 GiB0.01 GiB11±8.3%
Darkidol-Gemma-4-E4B-itQ8_08.0B7.48 GiB0.94 GiB8.99 GiB0.01 GiB11±8.3%
gemma-4-E4B-it-Claude-Opus-4.5-HERETIC-UNCENSORED-ThinkingQ8_08.0B7.48 GiB0.94 GiB8.99 GiB0.01 GiB11±8.3%
gemma-4-E4B-it-hereticQ8_08.0B7.48 GiB0.94 GiB8.99 GiB0.01 GiB11±8.3%
gemma-4-E4B-it-uncensoredQ8_08.0B7.48 GiB0.94 GiB8.99 GiB0.01 GiB11±8.3%
gemma-4-E4B-Agentic-Opus-Reasoning-GeminiCLI-mlx-4bitQ8_07.5B7.48 GiB0.94 GiB8.99 GiB0.01 GiB11±8.3%
OpenMedResearch-Gemma-4E4NQ8_08.0B7.48 GiB0.94 GiB8.99 GiB0.01 GiB11±8.3%
Reasoning-Medical0.1-E4B-sftQ8_08.0B7.48 GiB0.94 GiB8.99 GiB0.01 GiB11±8.3%
gemma-4-E4BQ8_08.0B7.48 GiB0.94 GiB8.99 GiB0.01 GiB11±8.3%
gemma-4-12B-itUD-IQ2_M12.0B3.92 GiB4.47 GiB8.99 GiB0.01 GiB11±8.3%
gemma-4-12B-it-hereticIQ2_M12.0B3.92 GiB4.47 GiB8.99 GiB0.01 GiB11±8.3%
InternVL3_5-14BQ4_K_M15.1B8.38 GiB0.00 GiB8.98 GiB0.02 GiB11±8.3%
Felldude-Uncensored-Ministral3-3B-bf16I1-Q4_K_S3.8B1.91 GiB6.50 GiB8.97 GiB0.03 GiB11±8.3%
Ministral-3-3B-Instruct-2512-BF16Q4_K_S4.3B1.91 GiB6.50 GiB8.97 GiB0.03 GiB11±8.3%
Amaretto-3BI1-Q4_K_S4.3B1.91 GiB6.50 GiB8.97 GiB0.03 GiB11±8.3%
Ministral-3-3B-Instruct-2512Q4_K_S3.8B1.91 GiB6.50 GiB8.97 GiB0.03 GiB11±8.3%
Ministral-3-3B-Reasoning-2512Q4_K_S4.3B1.91 GiB6.50 GiB8.97 GiB0.03 GiB11±8.3%
Llama-3.2-3B-Instruct-abliteratedI1-IQ3_XXS3.6B1.41 GiB7.00 GiB8.97 GiB0.03 GiB11±8.3%
HunyuanVideo-1.5Q8_08.3B8.38 GiB0.00 GiB8.97 GiB0.03 GiB12±8.3%
Fara1.5-9BQ5_K_M9.4B6.38 GiB2.00 GiB8.97 GiB0.03 GiB11±8.3%
QwenPaw-Flash-9BQ5_K_M9.4B6.38 GiB2.00 GiB8.97 GiB0.03 GiB11±8.3%
grug-9bQ5_K_M9.4B6.38 GiB2.00 GiB8.97 GiB0.03 GiB11±8.3%
OmniCoder-9BQ5_K_M9.4B6.38 GiB2.00 GiB8.97 GiB0.03 GiB11±8.3%
Ornith-1.0-9BQ5_K_M9.2B6.38 GiB2.00 GiB8.97 GiB0.03 GiB11±8.3%
Qwen3.5-9B-NeoQ5_K_M9.7B6.38 GiB2.00 GiB8.97 GiB0.03 GiB11±8.3%
gemma-4-E4B-it-Uncensored-MAXQ8_08.0B7.46 GiB0.94 GiB8.97 GiB0.03 GiB11±8.3%
Qwen3-15B-A2B-BaseMoEQ2_K15.6B5.41 GiB3.00 GiB8.96 GiB0.04 GiB15±37%
dolphincoder-starcoder2-15bKV unresolvedI1-IQ1_S16.0B3.31 GiB5.00 GiB8.96 GiB0.04 GiB12±8.3%
starcoder2-15bKV unresolvedIQ1_S16.0B3.31 GiB5.00 GiB8.96 GiB0.04 GiB12±8.3%
orpheus-3b-0.1-pretrainedIQ2_XXS3.8B1.40 GiB7.00 GiB8.96 GiB0.04 GiB11±8.3%
Qwen3-1.7BQ4_K_L2.0B1.41 GiB7.00 GiB8.95 GiB0.05 GiB11±8.3%
Llama-3.2-3B-Instruct-uncensoredQ2_K3.6B1.39 GiB7.00 GiB8.95 GiB0.05 GiB11±8.3%
nomic-embed-codeQ5_K_L7.1B4.84 GiB3.50 GiB8.95 GiB0.05 GiB12±8.3%
AfriqueGemma-12BI1-IQ2_XS12.2B3.88 GiB4.47 GiB8.95 GiB0.05 GiB12±8.3%
granite-20b-code-instruct-8kIQ3_S20.1B8.32 GiB0.00 GiB8.95 GiB0.05 GiB12±8.3%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a Apple M4 run?
802 of 2118 indexed open-weight models fit a Apple M4 at 65,536 context with f16 KV cache, the largest being gemma-4-12B at IQ2_S. That covers text, vision-language, image, video and speech models.
How much usable memory does a Apple M4 actually have?
Its nameplate is 12 GB, but about 8.37 GiB is available to a model once driver and compositor overhead is accounted for, and only 9 GB of the pool can be allocated to the GPU at all.
Is a Apple M4 fast for local AI?
Its memory bandwidth is 120 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.