Apple · apple

Apple M3 Ultra

Apple M3 Ultra has 96 GB of unified memory at 819 GB/s — about 66.96 GiB usable after driver and compositor overhead. 2071 of 2118 indexed models fit at 4K context with q4_0 KV. Note only 72 GB of its 96 GB is allocatable to the GPU.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
96 GB
LPDDR5-6400
Bandwidth
819 GB/s
1024-bit bus
Tensor FP16
—
dense
TDP
—
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 1781vision language 186image 2audio asr 39audio tts 21video 16embedding 26

What fits at 4K context

largest quantization that fits, per model · 2071 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
MiniMax-M2.7MoEIQ2_M229B71.17 GiB0.27 GiB71.98 GiB0.02 GiB44±37%
Mixtral-8x22B-Instruct-v0.1MoEIQ4_XS141B71.12 GiB0.25 GiB71.97 GiB0.03 GiB17±37%
Mixtral-8x22B-v0.1MoEIQ4_XS141B71.11 GiB0.25 GiB71.97 GiB0.03 GiB17±37%
Mixtral-8x22B-v0.1MoEIQ4_XS141B71.11 GiB0.25 GiB71.97 GiB0.03 GiB17±37%
Devstral-2-123B-Instruct-2512Q4_K_L125B70.87 GiB0.39 GiB71.96 GiB0.04 GiB9±8.3%
Qwen3.5-122B-A10B-hereticMoEI1-Q4_1123B71.35 GiB0.03 GiB71.96 GiB0.04 GiB45±37%
Step-3.5-Flash-REAP-121B-A11BI1-Q4_1121B70.61 GiB0.71 GiB71.90 GiB0.10 GiB9±8.3%
GLM-4.7-REAP-218B-A32BMoEUD-IQ2_M218B70.78 GiB0.40 GiB71.77 GiB0.23 GiB32±37%
GLM-4.5-Air-REAP-82B-A12BMoEQ6_K81.9B70.98 GiB0.20 GiB71.76 GiB0.24 GiB32±37%
GLM-4.6VMoEQ5_K_S108B70.79 GiB0.20 GiB71.57 GiB0.43 GiB36±37%
GLM-4.5VMoEI1-Q5_K_S108B70.53 GiB0.20 GiB71.31 GiB0.69 GiB36±37%
OYM-Qimi-122B-A10B-K2.6MoEI1-Q4_K_M125B70.64 GiB0.03 GiB71.24 GiB0.76 GiB46±37%
Qwopus3.5-122B-A10B-Kimi-K2.6-destill-healed-abliteratedMoEQ4_K_M123B70.63 GiB0.03 GiB71.24 GiB0.76 GiB46±37%
Apertus-70B-Instruct-2509Q8_070.6B69.87 GiB0.35 GiB70.95 GiB1.05 GiB9±8.3%
Meta-Llama-3-70B-InstructQ8_070.6B69.83 GiB0.35 GiB70.86 GiB1.14 GiB9±8.3%
calme-2.4-llama3-70bQ8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
calme-2.2-llama3-70bQ8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
L3.3-Electra-R1-70bQ8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
L3.3-70B-Magnum-v4-SEQ8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
Hermes-4-70BQ8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
Llama-3.3_70_b_uncensored_continuedQ8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
grok-oss-Revenant-70BQ8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
Llama-3.1-70BQ8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
Llama-3.3-70B-InstructQ8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
Llama-3.1-Nemotron-70B-Instruct-HFQ8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
Llama-3.3-70B-Instruct-abliteratedQ8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
Hermes-4-70B-hereticQ8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
Anubis-70B-v1.2Q8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
L3.3-70B-Euryale-v2.3Q8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
Rombos-LLM-70b-Llama-3.3Q8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
llama-3-firefunction-v2Q8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
DeepSeek-R1-Distill-Llama-70B-hereticQ8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
DeepSeek-R1-Distill-Llama-70BQ8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
DeepSeek-R1-Distill-Llama-70B-abliteratedQ8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
Legion-V2.1-LLaMa-70BQ8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
Tess-R1-Limerick-Llama-3.1-70BQ8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
Assistant_Pepe_70BQ8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
SEMIKONG-70BQ8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
functionary-medium-v3.2KV unresolvedQ8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
Llama-3.1-WhiteRabbitNeo-2-70BQ8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
Athene-70BQ8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
Infinity-Instruct-7M-Gen-Llama3_1-70BQ8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
Hermes-3-Llama-3.1-70BQ8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
L3.3-70B-Magnum-DiamondQ8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
Meta-Llama-3-70B-Instruct-abliterated-v3.5Q8_070.6B69.83 GiB0.35 GiB70.85 GiB1.15 GiB9±8.3%
Mistral-Medium-3.5-128BI1-Q4_K_M128B69.75 GiB0.39 GiB70.85 GiB1.15 GiB10±8.3%
XORTRON-NXTXPRTXXLI1-Q4_K_M128B69.75 GiB0.39 GiB70.85 GiB1.15 GiB10±8.3%
MiniMax-M2.7-BF16-ultra-uncensored-hereticMoEI1-IQ2_M229B69.70 GiB0.27 GiB70.50 GiB1.50 GiB45±37%
MiniMax-M2.1MoEI1-IQ2_M229B69.70 GiB0.27 GiB70.50 GiB1.50 GiB45±37%
MiniMax-M2.5MoEI1-IQ2_M229B69.70 GiB0.27 GiB70.50 GiB1.50 GiB45±37%
dots.llm1.instMoEQ3_K_M143B68.74 GiB1.09 GiB70.41 GiB1.59 GiB38±37%
Qwen3.5-122B-A10BMoEQ4_K_S125B69.66 GiB0.03 GiB70.27 GiB1.73 GiB46±37%
Ornith-1.0-35B-AEON-Ultimate-Uncensored-BF16MoEQ8_035.1B69.57 GiB0.02 GiB70.15 GiB1.85 GiB46±37%
GLM-4.6-REAP-268B-A32BMoEUD-IQ1_S269B69.09 GiB0.40 GiB70.08 GiB1.92 GiB36±37%
MiniMax-M2.1-REAP-139B-A10BMoEI1-IQ4_XS139B69.15 GiB0.27 GiB69.95 GiB2.05 GiB39±37%
m51Lab-MiniMax-M2.7-REAP-139B-A10BMoEI1-IQ4_XS139B69.15 GiB0.27 GiB69.95 GiB2.05 GiB39±37%
Llama-4-Scout-17B-16E-InstructMoEKV unresolvedQ5_K_S109B69.16 GiB0.21 GiB69.94 GiB2.06 GiB37±37%
Step-3.7-FlashUD-IQ3_XXS201B68.54 GiB0.71 GiB69.83 GiB2.17 GiB10±8.3%
MiniMax-M2MoEUD-IQ2_XXS229B68.92 GiB0.27 GiB69.73 GiB2.27 GiB45±37%
GLM-4.5-Air-DerestrictedMoEQ4_K_L110B68.88 GiB0.20 GiB69.66 GiB2.34 GiB37±37%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Measured on this card

third-party benchmarks, aggregated
Workload◍MedianMiddle 50%Runs
Prompt processing1471.24 tok/s1116.66–1488.1811
Text generation64.16 tok/s52.90–92.0411
Benchmarked· n=11

Aggregated from community-submitted runs, so the spread is wide by nature — it covers different models, resolutions, step counts and settings, not one controlled configuration. Read the middle 50% rather than the median alone. These figures are reproduced with attribution from llama.cpp-discussion-10879.

Questions people ask

What AI models can a Apple M3 Ultra run?
2071 of 2118 indexed open-weight models fit a Apple M3 Ultra at 4,096 context with q4_0 KV cache, the largest being MiniMax-M2.7 at IQ2_M. That covers text, vision-language, image, video and speech models.
How much usable memory does a Apple M3 Ultra actually have?
Its nameplate is 96 GB, but about 66.96 GiB is available to a model once driver and compositor overhead is accounted for, and only 72 GB of the pool can be allocated to the GPU at all.
Is a Apple M3 Ultra fast for local AI?
Its memory bandwidth is 819 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.