Apple · apple

Apple M3 Max

Apple M3 Max has 48 GB of unified memory at 410 GB/s — about 33.48 GiB usable after driver and compositor overhead. 1891 of 2118 indexed models fit at 128K context with q8_0 KV. Note only 36 GB of its 48 GB is allocatable to the GPU.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
48 GB
LPDDR5-6400
Bandwidth
410 GB/s
512-bit bus
Tensor FP16
—
dense
TDP
—
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 1610image 2vision language 177audio asr 39audio tts 21video 16embedding 26

What fits at 128K context

largest quantization that fits, per model · 1891 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
deepseek-coder-6.7B-kexerI1-IQ1_S6.7B1.42 GiB34.00 GiB36.00 GiB0.00 GiB9±8.3%
Magicoder-S-DS-6.7BI1-IQ1_S6.7B1.42 GiB34.00 GiB36.00 GiB0.00 GiB9±8.3%
deepseek-coder-6.7b-baseI1-IQ1_S6.7B1.42 GiB34.00 GiB36.00 GiB0.00 GiB9±8.3%
WizardLM-7B-UncensoredI1-IQ1_S6.7B1.42 GiB34.00 GiB36.00 GiB0.00 GiB9±8.3%
Llama-2-7B-32K-InstructI1-IQ1_S6.7B1.42 GiB34.00 GiB36.00 GiB0.00 GiB9±8.3%
Luna-AI-Llama2-UncensoredI1-IQ1_S6.7B1.42 GiB34.00 GiB36.00 GiB0.00 GiB9±8.3%
Swallow-7b-NVE-instruct-hfI1-IQ1_S6.7B1.42 GiB34.00 GiB36.00 GiB0.00 GiB9±8.3%
magnum-v2-32bQ4_K_M32.5B18.35 GiB17.00 GiB35.99 GiB0.01 GiB9±8.3%
Qwen3.5-88BMoEI1-IQ3_XS87.7B33.82 GiB1.59 GiB35.99 GiB0.01 GiB32±37%
Mistral-Small-Instruct-2409IQ1_M22.2B20.51 GiB14.88 GiB35.99 GiB0.01 GiB9±8.3%
Janus-Pro-7BI1-IQ4_XS7.4B3.54 GiB31.88 GiB35.99 GiB0.01 GiB9±8.3%
deepseek-coder-7b-instruct-v1.5I1-IQ4_XS6.9B3.54 GiB31.88 GiB35.99 GiB0.01 GiB9±8.3%
Qwen3.5-122B-A10BMoEIQ2_XXS125B33.81 GiB1.59 GiB35.98 GiB0.02 GiB34±37%
Skyfall-31B-v4.2Q5_K_M31.4B20.97 GiB14.34 GiB35.98 GiB0.02 GiB9±8.3%
c4ai-command-r-08-2024Q6_K32.3B24.68 GiB10.63 GiB35.97 GiB0.03 GiB9±8.3%
Hermes-4.3-36B-hereticIQ4_XS36.2B18.32 GiB17.00 GiB35.97 GiB0.03 GiB9±8.3%
Noromaid-v0.4-Mixtral-Instruct-8x7b-ZlossMoEQ4_K_M46.7B26.88 GiB8.50 GiB35.97 GiB0.03 GiB11±37%
gemma-7bI1-Q5_K_S8.5B5.57 GiB29.75 GiB35.94 GiB0.06 GiB9±8.3%
deepseek-math-7b-instructQ3_K_L6.9B3.49 GiB31.88 GiB35.94 GiB0.06 GiB9±8.3%
deepseek-llm-7b-chatQ3_K_L6.9B3.49 GiB31.88 GiB35.94 GiB0.06 GiB9±8.3%
Qwen3.5-122B-A10B-hereticMoEI1-IQ2_S123B33.77 GiB1.59 GiB35.94 GiB0.06 GiB34±37%
Qwen2.5-Coder-14B-InstructQ6_K14.8B22.58 GiB12.75 GiB35.93 GiB0.07 GiB9±8.3%
Hunyuan-A13B-InstructMoEUD-IQ2_M80.4B26.85 GiB8.50 GiB35.90 GiB0.10 GiB9±8.3%
Phi-3.5-MoE-instructMoEKV unresolvedQ5_K_S41.9B26.84 GiB8.50 GiB35.89 GiB0.11 GiB14±37%
MN-GRAND-23.5B-Gutenberg-UNCENSORED-V2-GLM4.7-ThinkingI1-Q4_123.4B13.77 GiB21.52 GiB35.88 GiB0.12 GiB9±8.3%
Seed-OSS-36B-InstructIQ4_XS36.2B18.18 GiB17.00 GiB35.83 GiB0.17 GiB9±8.3%
granite-3.1-8b-instructQ8_08.2B24.62 GiB10.63 GiB35.82 GiB0.18 GiB9±8.3%
Yi-34B-200K-DARE-megamerge-v8I1-Q4_K_M34.4B19.24 GiB15.94 GiB35.81 GiB0.19 GiB9±8.3%
dolphin-2.9.1-yi-1.5-34b-hereticQ4_K_M34.4B19.24 GiB15.94 GiB35.81 GiB0.19 GiB9±8.3%
dolphin-2.9.1-yi-1.5-34bI1-Q4_K_M34.4B19.24 GiB15.94 GiB35.81 GiB0.19 GiB9±8.3%
OrionStar-Yi-34B-Chat-LlamaI1-Q4_K_M34.4B19.24 GiB15.94 GiB35.81 GiB0.19 GiB9±8.3%
Yi-34B-200K-LlamafiedI1-Q4_K_M34.4B19.24 GiB15.94 GiB35.81 GiB0.19 GiB9±8.3%
Yi-1.5-34BQ4_K_M34.4B19.24 GiB15.94 GiB35.81 GiB0.19 GiB9±8.3%
Nous-Hermes-2-Yi-34BQ4_K_M34.4B19.24 GiB15.94 GiB35.81 GiB0.19 GiB9±8.3%
Merged-RP-Stew-V2-34BI1-Q4_K_M34.4B19.24 GiB15.94 GiB35.81 GiB0.19 GiB9±8.3%
Capybara-Tess-Yi-34B-200KQ4_K_M34.4B19.24 GiB15.94 GiB35.81 GiB0.19 GiB9±8.3%
Nous-Capybara-limarpv3-34BQ4_K_M34.4B19.24 GiB15.94 GiB35.81 GiB0.19 GiB9±8.3%
Qwen3-48B-A4B-Savant-Commander-Distill-12X-Closed-Open-Heretic-UncensoredMoEI1-Q6_K33.6B25.69 GiB9.56 GiB35.81 GiB0.19 GiB12±37%
Seed-OSS-36B-Instruct-biprojected-norm-preserving-abliteratedI1-IQ4_XS36.2B18.16 GiB17.00 GiB35.81 GiB0.19 GiB9±8.3%
Hermes-4.3-36BIQ4_XS36.2B18.16 GiB17.00 GiB35.81 GiB0.19 GiB9±8.3%
OLMo-2-0325-32BQ4_K_M32.2B18.14 GiB17.00 GiB35.79 GiB0.21 GiB9±8.3%
llm-surgery-dark-arts-gpt-oss-60b-96a12MoEI1-Q3_K_M60.9B33.63 GiB1.60 GiB35.77 GiB0.23 GiB23±37%
Skyfall-31B-v4.2-hereticI1-Q5_K_M31.4B20.72 GiB14.34 GiB35.74 GiB0.26 GiB10±8.3%
Mixtral_34Bx2_MoE_60BMoEQ2_K60.8B19.14 GiB15.94 GiB35.71 GiB0.29 GiB6±37%
xLAM-8x7b-rMoEQ4_K_L46.7B26.59 GiB8.50 GiB35.67 GiB0.33 GiB11±37%
deepseek-coder-33b-instructQ4_K_M33.3B18.57 GiB16.47 GiB35.67 GiB0.33 GiB10±8.3%
deepseek-coder-33b-baseQ4_K_M33.3B18.57 GiB16.47 GiB35.67 GiB0.33 GiB10±8.3%
WhiteRabbitNeo-33B-v1Q4_K_M33.3B18.57 GiB16.47 GiB35.67 GiB0.33 GiB10±8.3%
IQuest-Coder-V1-40B-InstructI1-Q2_K39.8B13.76 GiB21.25 GiB35.65 GiB0.35 GiB10±8.3%
WizardCoder-Python-34B-V1.0I1-Q5_K_M33.7B22.20 GiB12.75 GiB35.60 GiB0.40 GiB10±8.3%
Phind-CodeLlama-34B-Python-v1I1-Q5_K_M33.7B22.20 GiB12.75 GiB35.60 GiB0.40 GiB10±8.3%
Phind-CodeLlama-34B-v2I1-Q5_K_M33.7B22.20 GiB12.75 GiB35.60 GiB0.40 GiB10±8.3%
CodeLlama-34b-instruct-hfQ5_K_M33.7B22.20 GiB12.75 GiB35.60 GiB0.40 GiB10±8.3%
WizardLM-1.0-Uncensored-CodeLlama-34bQ5_K_M33.7B22.20 GiB12.75 GiB35.60 GiB0.40 GiB10±8.3%
dolphin-2.6-mixtral-8x7bMoEI1-Q4_K_M46.7B26.49 GiB8.50 GiB35.58 GiB0.42 GiB11±37%
Nous-Hermes-2-Mixtral-8x7B-DPOMoEQ4_K_M46.7B26.49 GiB8.50 GiB35.58 GiB0.42 GiB11±37%
Mixtral-8x7B-Instruct-v0.1MoEQ4_K_M46.7B26.49 GiB8.50 GiB35.58 GiB0.42 GiB11±37%
dolphin-2.5-mixtral-8x7bMoEQ4_K_M46.7B26.49 GiB8.50 GiB35.58 GiB0.42 GiB11±37%
Mixtral-8x7B-v0.1MoEQ4_K_M46.7B26.49 GiB8.50 GiB35.58 GiB0.42 GiB11±37%
Ling-liteMoEBF1616.8B31.31 GiB3.72 GiB35.57 GiB0.43 GiB21±37%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a Apple M3 Max run?
1891 of 2118 indexed open-weight models fit a Apple M3 Max at 131,072 context with q8_0 KV cache, the largest being deepseek-coder-6.7B-kexer at I1-IQ1_S. That covers text, vision-language, image, video and speech models.
How much usable memory does a Apple M3 Max actually have?
Its nameplate is 48 GB, but about 33.48 GiB is available to a model once driver and compositor overhead is accounted for, and only 36 GB of the pool can be allocated to the GPU at all.
Is a Apple M3 Max fast for local AI?
Its memory bandwidth is 410 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.