Apple · apple

Apple M3 Ultra

Apple M3 Ultra has 96 GB of unified memory at 819 GB/s — about 66.96 GiB usable after driver and compositor overhead. 2071 of 2118 indexed models fit at 8K context with q8_0 KV. Note only 72 GB of its 96 GB is allocatable to the GPU.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
96 GB
LPDDR5-6400
Bandwidth
819 GB/s
1024-bit bus
Tensor FP16
—
dense
TDP
—
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 1781vision language 186image 2audio asr 39audio tts 21video 16embedding 26

What fits at 8K context

largest quantization that fits, per model · 2071 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
Apertus-70B-Instruct-2509Q8_070.6B69.87 GiB1.33 GiB71.93 GiB0.07 GiB9±8.3%
Devstral-2-123B-Instruct-2512Q4_K_M125B69.75 GiB1.46 GiB71.92 GiB0.08 GiB9±8.3%
Mistral-Medium-3.5-128BI1-Q4_K_M128B69.75 GiB1.46 GiB71.92 GiB0.08 GiB9±8.3%
XORTRON-NXTXPRTXXLI1-Q4_K_M128B69.75 GiB1.46 GiB71.92 GiB0.08 GiB9±8.3%
GLM-4.5VMoEI1-Q5_K_S108B70.53 GiB0.76 GiB71.87 GiB0.13 GiB35±37%
Meta-Llama-3-70B-InstructQ8_070.6B69.83 GiB1.33 GiB71.84 GiB0.16 GiB9±8.3%
calme-2.4-llama3-70bQ8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
calme-2.2-llama3-70bQ8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
L3.3-Electra-R1-70bQ8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
L3.3-70B-Magnum-v4-SEQ8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
Hermes-4-70BQ8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
Llama-3.3_70_b_uncensored_continuedQ8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
grok-oss-Revenant-70BQ8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
Llama-3.1-70BQ8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
Llama-3.3-70B-InstructQ8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
Llama-3.1-Nemotron-70B-Instruct-HFQ8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
Llama-3.3-70B-Instruct-abliteratedQ8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
Hermes-4-70B-hereticQ8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
Anubis-70B-v1.2Q8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
L3.3-70B-Euryale-v2.3Q8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
Rombos-LLM-70b-Llama-3.3Q8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
llama-3-firefunction-v2Q8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
DeepSeek-R1-Distill-Llama-70B-hereticQ8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
DeepSeek-R1-Distill-Llama-70BQ8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
DeepSeek-R1-Distill-Llama-70B-abliteratedQ8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
Legion-V2.1-LLaMa-70BQ8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
Tess-R1-Limerick-Llama-3.1-70BQ8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
Assistant_Pepe_70BQ8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
SEMIKONG-70BQ8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
functionary-medium-v3.2KV unresolvedQ8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
Llama-3.1-WhiteRabbitNeo-2-70BQ8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
Athene-70BQ8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
Infinity-Instruct-7M-Gen-Llama3_1-70BQ8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
Hermes-3-Llama-3.1-70BQ8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
L3.3-70B-Magnum-DiamondQ8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
Meta-Llama-3-70B-Instruct-abliterated-v3.5Q8_070.6B69.83 GiB1.33 GiB71.83 GiB0.17 GiB9±8.3%
OYM-Qimi-122B-A10B-K2.6MoEI1-Q4_K_M125B70.64 GiB0.10 GiB71.31 GiB0.69 GiB45±37%
Qwopus3.5-122B-A10B-Kimi-K2.6-destill-healed-abliteratedMoEQ4_K_M123B70.63 GiB0.10 GiB71.31 GiB0.69 GiB45±37%
MiniMax-M2.7-BF16-ultra-uncensored-hereticMoEI1-IQ2_M229B69.70 GiB1.03 GiB71.26 GiB0.74 GiB41±37%
MiniMax-M2.1MoEI1-IQ2_M229B69.70 GiB1.03 GiB71.26 GiB0.74 GiB41±37%
MiniMax-M2.5MoEI1-IQ2_M229B69.70 GiB1.03 GiB71.26 GiB0.74 GiB41±37%
Step-3.7-FlashUD-IQ3_XXS201B68.54 GiB2.14 GiB71.26 GiB0.74 GiB9±8.3%
GLM-4.6-REAP-268B-A32BMoEUD-IQ1_S269B69.09 GiB1.53 GiB71.21 GiB0.79 GiB32±37%
MiniMax-M2.1-REAP-139B-A10BMoEI1-IQ4_XS139B69.15 GiB1.03 GiB70.71 GiB1.29 GiB37±37%
m51Lab-MiniMax-M2.7-REAP-139B-A10BMoEI1-IQ4_XS139B69.15 GiB1.03 GiB70.71 GiB1.29 GiB37±37%
Behemoth-X-123B-v2Q4_K_L123B68.47 GiB1.46 GiB70.63 GiB1.37 GiB10±8.3%
Step-3.5-Flash-REAP-121B-A11BI1-Q4_K_M121B67.82 GiB2.14 GiB70.54 GiB1.46 GiB10±8.3%
Llama-4-Scout-17B-16E-InstructMoEKV unresolvedQ5_K_S109B69.16 GiB0.80 GiB70.53 GiB1.47 GiB35±37%
MiniMax-M2MoEUD-IQ2_XXS229B68.92 GiB1.03 GiB70.48 GiB1.52 GiB42±37%
c4ai-command-r-plus-08-2024Q5_K_M104B68.57 GiB1.06 GiB70.36 GiB1.64 GiB10±8.3%
Mistral-Large-Instruct-2411Q4_K_M123B68.19 GiB1.46 GiB70.36 GiB1.64 GiB10±8.3%
Qwen3.5-122B-A10BMoEQ4_K_S125B69.66 GiB0.10 GiB70.34 GiB1.66 GiB46±37%
GLM-4.5-Air-DerestrictedMoEQ4_K_L110B68.88 GiB0.76 GiB70.22 GiB1.78 GiB35±37%
Ornith-1.0-35B-AEON-Ultimate-Uncensored-BF16MoEQ8_035.1B69.57 GiB0.08 GiB70.21 GiB1.79 GiB46±37%
Laguna-S-2.1MoEQ4_1118B68.96 GiB0.27 GiB69.81 GiB2.19 GiB43±37%
GLM-4.5-AirMoEQ4_K_M110B68.45 GiB0.76 GiB69.80 GiB2.20 GiB35±37%
Qwen3.5-122B-A10B-hereticMoEI1-Q4_K_M123B69.11 GiB0.10 GiB69.79 GiB2.21 GiB46±37%
HunyuanImage-2.1Q6_K17.5B68.97 GiB0.00 GiB69.57 GiB2.43 GiB10±8.3%
MiMo-V2-FlashMoEKV unresolvedIQ2_XXS310B68.47 GiB0.50 GiB69.57 GiB2.43 GiB44±37%
dots.llm1.instMoEQ3_K_S143B64.78 GiB4.12 GiB69.48 GiB2.52 GiB30±37%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Measured on this card

third-party benchmarks, aggregated
Workload◍MedianMiddle 50%Runs
Prompt processing1471.24 tok/s1116.66–1488.1811
Text generation64.16 tok/s52.90–92.0411
Benchmarked· n=11

Aggregated from community-submitted runs, so the spread is wide by nature — it covers different models, resolutions, step counts and settings, not one controlled configuration. Read the middle 50% rather than the median alone. These figures are reproduced with attribution from llama.cpp-discussion-10879.

Questions people ask

What AI models can a Apple M3 Ultra run?
2071 of 2118 indexed open-weight models fit a Apple M3 Ultra at 8,192 context with q8_0 KV cache, the largest being Apertus-70B-Instruct-2509 at Q8_0. That covers text, vision-language, image, video and speech models.
How much usable memory does a Apple M3 Ultra actually have?
Its nameplate is 96 GB, but about 66.96 GiB is available to a model once driver and compositor overhead is accounted for, and only 72 GB of the pool can be allocated to the GPU at all.
Is a Apple M3 Ultra fast for local AI?
Its memory bandwidth is 819 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.