Apple · apple

Apple M5

Apple M5 has 12 GB of unified memory at 154 GB/s — about 8.37 GiB usable after driver and compositor overhead. 1428 of 2118 indexed models fit at 32K context with q8_0 KV. Note only 9 GB of its 12 GB is allocatable to the GPU.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
12 GB
LPDDR5X-9600
Bandwidth
154 GB/s
128-bit bus
Tensor FP16
—
dense
TDP
—
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 1220video 12vision language 110embedding 26audio asr 38audio tts 21image 1

What fits at 32K context

largest quantization that fits, per model · 1428 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
ERNIE-21B-A3B-Thinking-Gemini-3-Pro-High-Reasoning-V2I1-Q2_K21.8B7.50 GiB0.93 GiB9.00 GiB0.00 GiB14±8.3%
ERNIE-21B-A3B-Claude-4.5-High-OPUS-ThinkingI1-Q2_K21.8B7.50 GiB0.93 GiB9.00 GiB0.00 GiB14±8.3%
ERNIE-4.5-21B-A3B-ThinkingI1-Q2_K21.8B7.50 GiB0.93 GiB9.00 GiB0.00 GiB14±8.3%
Wan2.1-T2V-14BQ4_014.3B8.41 GiB0.00 GiB9.00 GiB0.00 GiB14±8.3%
Apriel-1.6-15b-ThinkerI1-Q2_K14.9B5.21 GiB3.19 GiB8.99 GiB0.01 GiB14±8.3%
Qwen3.5-14B-A3B-Claude-4.6-Opus-Reasoning-Distilled-reapMoEQ4_K_M14.1B8.10 GiB0.33 GiB8.98 GiB0.02 GiB39±37%
gemma-4-12B-it-qat-q4_0-unquantized-uncensored-hereticQ4_012.0B7.07 GiB1.31 GiB8.98 GiB0.02 GiB14±8.3%
NVIDIA-Nemotron-Nano-9B-v2Q2_K8.9B4.66 GiB3.72 GiB8.98 GiB0.02 GiB14±8.3%
openNemo-9B-abliteratedQ2_K8.9B4.66 GiB3.72 GiB8.98 GiB0.02 GiB14±8.3%
InternVL3_5-14BQ4_K_M15.1B8.38 GiB0.00 GiB8.98 GiB0.02 GiB14±8.3%
Gemma-4-12B-StyleTuneI1-Q4_K_S13.0B7.07 GiB1.31 GiB8.98 GiB0.02 GiB14±8.3%
gemma-4-12b-heretic-styletune-headI1-Q4_K_S12.0B7.07 GiB1.31 GiB8.98 GiB0.02 GiB14±8.3%
syrian-gemma-12bI1-Q4_K_S13.0B7.07 GiB1.31 GiB8.98 GiB0.02 GiB14±8.3%
Llama-3.2-8X3B-MOE-Dark-Champion-Instruct-uncensored-abliterated-18.4BMoEQ2_K18.4B6.56 GiB1.86 GiB8.98 GiB0.02 GiB18±37%
dolphin-2.9.2-Phi-3-MediumKV unresolvedIQ3_XXS14.0B5.05 GiB3.32 GiB8.98 GiB0.02 GiB14±8.3%
ERNIE-4.5-21B-A3B-PTUD-IQ2_M21.9B7.47 GiB0.93 GiB8.97 GiB0.03 GiB14±8.3%
Phi-3.5-mini-instructQ4_K_S3.8B2.04 GiB6.38 GiB8.97 GiB0.03 GiB14±8.3%
HunyuanVideo-1.5Q8_08.3B8.38 GiB0.00 GiB8.97 GiB0.03 GiB14±8.3%
NuExtract-1.5Q4_K_S3.8B2.04 GiB6.38 GiB8.97 GiB0.03 GiB14±8.3%
Phi-3.5-mini-instructQ4_K_S3.8B2.04 GiB6.38 GiB8.97 GiB0.03 GiB14±8.3%
Phi-3.5-mini-instruct_UncensoredQ4_K_S3.8B2.04 GiB6.38 GiB8.97 GiB0.03 GiB14±8.3%
Phi-3-mini-128k-instructQ4_K_S3.8B2.04 GiB6.38 GiB8.97 GiB0.03 GiB14±8.3%
Phi-3-mini-4k-instructQ4_K_S3.8B2.04 GiB6.38 GiB8.97 GiB0.03 GiB14±8.3%
octo-netQ4_K_S3.8B2.04 GiB6.38 GiB8.97 GiB0.03 GiB14±8.3%
reka-flash-3.1I1-IQ1_S20.9B6.15 GiB2.19 GiB8.97 GiB0.03 GiB14±8.3%
Snowpiercer-15B-v4-hereticI1-Q2_K_S15.0B5.05 GiB3.32 GiB8.96 GiB0.04 GiB14±8.3%
internlm2-math-plus-20bI1-IQ2_XXS19.9B5.16 GiB3.19 GiB8.96 GiB0.04 GiB14±8.3%
granite-3.3-8b-instructQ5_18.2B5.72 GiB2.66 GiB8.96 GiB0.04 GiB14±8.3%
granite-3.2-8b-instructQ5_18.2B5.72 GiB2.66 GiB8.96 GiB0.04 GiB14±8.3%
Nanbeige4.1-3BF163.9B7.33 GiB1.06 GiB8.96 GiB0.04 GiB14±8.3%
GLM-4.6V-FlashQ6_K10.3B7.70 GiB0.66 GiB8.95 GiB0.05 GiB14±8.3%
GLM-Z1-9B-0414Q6_K9.4B7.70 GiB0.66 GiB8.95 GiB0.05 GiB14±8.3%
glm4.1v-9b-base-sftI1-Q6_K10.3B7.70 GiB0.66 GiB8.95 GiB0.05 GiB14±8.3%
GLM-4-9B-0414Q6_K9.4B7.70 GiB0.66 GiB8.95 GiB0.05 GiB14±8.3%
GLM-4.1V-9B-ThinkingQ6_K10.3B7.70 GiB0.66 GiB8.95 GiB0.05 GiB14±8.3%
granite-3.1-8b-instructQ5_18.2B5.71 GiB2.66 GiB8.95 GiB0.05 GiB14±8.3%
gemma-3-12b-it-vl-Gemini-3-Pro-Preview-Heretic-Uncensored-ThinkingI1-Q4_112.2B7.04 GiB1.31 GiB8.95 GiB0.05 GiB14±8.3%
gemma-3-12b-it-vl-Deepseek-v3.1-Heretic-Uncensored-ThinkingI1-Q4_112.2B7.04 GiB1.31 GiB8.95 GiB0.05 GiB14±8.3%
gemma-3-12b-it-vl-GLM-4.7-Flash-Heretic-Uncensored-ThinkingI1-Q4_112.2B7.04 GiB1.31 GiB8.95 GiB0.05 GiB14±8.3%
Floppa-12B-Gemma3-UncensoredI1-Q4_112.2B7.04 GiB1.31 GiB8.95 GiB0.05 GiB14±8.3%
gemma-3-12b-it-hereticI1-Q4_112.2B7.04 GiB1.31 GiB8.95 GiB0.05 GiB14±8.3%
gemma-3-12b-it-abliteratedQ4_112.2B7.04 GiB1.31 GiB8.95 GiB0.05 GiB14±8.3%
gemma-3-12b-itQ4_112.2B7.04 GiB1.31 GiB8.95 GiB0.05 GiB14±8.3%
granite-20b-code-instruct-8kIQ3_S20.1B8.32 GiB0.00 GiB8.95 GiB0.05 GiB15±8.3%
granite-20b-code-base-8kI1-IQ3_S20.1B8.32 GiB0.00 GiB8.95 GiB0.05 GiB15±8.3%
Ministral-3-14B-Instruct-2512-BF16-abliteratedI1-IQ3_S13.9B5.68 GiB2.66 GiB8.95 GiB0.05 GiB14±8.3%
Ministral-3-14B-Reasoning-2512-UncensoredI1-IQ3_S13.9B5.68 GiB2.66 GiB8.95 GiB0.05 GiB14±8.3%
Ling-liteMoEQ3_K_S16.8B7.47 GiB0.93 GiB8.95 GiB0.05 GiB30±37%
DeepSeek-Coder-V2-Lite-BaseMoEI1-Q3_K_L15.7B7.88 GiB0.50 GiB8.94 GiB0.06 GiB35±37%
DeepSeek-Coder-V2-Lite-InstructMoEQ3_K_L15.7B7.88 GiB0.50 GiB8.94 GiB0.06 GiB35±37%
DeepSeek-V2-Lite-ChatMoEQ3_K_L15.7B7.88 GiB0.50 GiB8.94 GiB0.06 GiB35±37%
DeepSeek-V2-Lite-Chat-Uncensored-Unbiased-ReasonerMoEQ3_K_L15.7B7.88 GiB0.50 GiB8.94 GiB0.06 GiB35±37%
Qwen3.5-4B-Claude-4.6-OS-Auto-Variable-HERETIC-UNCENSORED-THINKINGF164.5B7.85 GiB0.53 GiB8.94 GiB0.06 GiB14±8.3%
Agents-A1-4B-Heretic-ARA-Refusals8BF164.5B7.85 GiB0.53 GiB8.94 GiB0.06 GiB14±8.3%
Agents-A1-4BF164.5B7.85 GiB0.53 GiB8.94 GiB0.06 GiB14±8.3%
Qwen3.5-4B-NSFW-ARA-Heretic-LiteroticaF164.2B7.85 GiB0.53 GiB8.94 GiB0.06 GiB14±8.3%
Qwen3.5-4B-SOMPOA-heresy-v2F164.5B7.85 GiB0.53 GiB8.94 GiB0.06 GiB14±8.3%
Huihui-Qwen3.5-4B-abliteratedF164.5B7.85 GiB0.53 GiB8.94 GiB0.06 GiB14±8.3%
Qwen3.5-4B-Safety-ThinkingF164.2B7.85 GiB0.53 GiB8.94 GiB0.06 GiB14±8.3%
GRaPE-2-MiniF164.7B7.85 GiB0.53 GiB8.94 GiB0.06 GiB14±8.3%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Measured on this card

third-party benchmarks, aggregated
Workload◍MedianMiddle 50%Runs
Prompt processing489.78 tok/s264.15–636.369
Text generation16.62 tok/s9.67–27.929
Benchmarked· n=9

Aggregated from community-submitted runs, so the spread is wide by nature — it covers different models, resolutions, step counts and settings, not one controlled configuration. Read the middle 50% rather than the median alone. These figures are reproduced with attribution from llama.cpp-discussion-4167.

Questions people ask

What AI models can a Apple M5 run?
1428 of 2118 indexed open-weight models fit a Apple M5 at 32,768 context with q8_0 KV cache, the largest being ERNIE-21B-A3B-Thinking-Gemini-3-Pro-High-Reasoning-V2 at I1-Q2_K. That covers text, vision-language, image, video and speech models.
How much usable memory does a Apple M5 actually have?
Its nameplate is 12 GB, but about 8.37 GiB is available to a model once driver and compositor overhead is accounted for, and only 9 GB of the pool can be allocated to the GPU at all.
Is a Apple M5 fast for local AI?
Its memory bandwidth is 154 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.
Apple M5 — what AI models can it run locally? — ossmodeldb