AMD · datacenter

Instinct MI60

Instinct MI60 has 32 GB of VRAM at 1024 GB/s — about 29.76 GiB usable after driver and compositor overhead. 1995 of 2118 indexed models fit at 64K context with q4_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
32 GB
HBM2
Bandwidth
1024 GB/s
4096-bit bus
Tensor FP16
—
dense
TDP
300 W
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 1711vision language 180video 16image 2embedding 26audio asr 39audio tts 21

What fits at 64K context

largest quantization that fits, per model · 1995 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
Qwen3-Next-80B-A3B-ThinkingMoEQ2_K81.3B27.17 GiB1.69 GiB29.74 GiB0.02 GiB87±37%
Qwen3-Next-80B-A3B-InstructMoEQ2_K81.3B27.17 GiB1.69 GiB29.74 GiB0.02 GiB87±37%
codegeex4-all-9bF169.4B17.52 GiB11.25 GiB29.71 GiB0.05 GiB22±26.5%
glm-4-9b-chat-abliteratedF169.4B17.52 GiB11.25 GiB29.71 GiB0.05 GiB22±26.5%
glm-4-9b-chatBF169.4B17.52 GiB11.25 GiB29.71 GiB0.05 GiB22±26.5%
CalmeRys-78B-Orpo-v0.1I1-IQ1_S78.0B22.62 GiB6.05 GiB29.70 GiB0.06 GiB22±26.5%
Kimi-Linear-48B-A3B-InstructMoEQ4_K_L49.1B28.26 GiB0.53 GiB29.70 GiB0.06 GiB22±26.5%
Salience-1.5-ProMoEQ6_K36.0B28.43 GiB0.35 GiB29.69 GiB0.07 GiB110±37%
Qwable-v1MoEQ6_K36.0B28.43 GiB0.35 GiB29.69 GiB0.07 GiB110±37%
T-SearchMoEQ6_K36.0B28.43 GiB0.35 GiB29.69 GiB0.07 GiB110±37%
dolphin-2.6-mixtral-8x7bMoEI1-Q4_K_M46.7B26.49 GiB2.25 GiB29.68 GiB0.08 GiB36±37%
Nous-Hermes-2-Mixtral-8x7B-DPOMoEQ4_K_M46.7B26.49 GiB2.25 GiB29.68 GiB0.08 GiB36±37%
Mixtral-8x7B-Instruct-v0.1MoEQ4_K_M46.7B26.49 GiB2.25 GiB29.68 GiB0.08 GiB36±37%
xLAM-8x7b-rMoEQ4_K_M46.7B26.49 GiB2.25 GiB29.68 GiB0.08 GiB36±37%
dolphin-2.5-mixtral-8x7bMoEQ4_K_M46.7B26.49 GiB2.25 GiB29.68 GiB0.08 GiB36±37%
Mixtral-8x7B-v0.1MoEQ4_K_M46.7B26.49 GiB2.25 GiB29.68 GiB0.08 GiB36±37%
Bernini-RQ8_014.3B28.71 GiB0.00 GiB29.66 GiB0.10 GiB22±26.5%
Magistral-Small-2509-VisionQ6_K_L24.0B25.83 GiB2.81 GiB29.66 GiB0.10 GiB23±26.5%
Noromaid-20b-v0.1.1I1-Q2_K20.0B6.91 GiB21.80 GiB29.65 GiB0.11 GiB22±26.5%
v6-Finch-14B-HFQ6_K_L14.1B11.55 GiB17.16 GiB29.65 GiB0.11 GiB22±26.5%
Qwen3.5-88BMoEI1-Q2_K_S87.7B28.30 GiB0.42 GiB29.65 GiB0.11 GiB98±37%
Gemma-The-Writer-N-Restless-Quill-10B-UncensoredQ6_K10.0B25.24 GiB3.46 GiB29.65 GiB0.11 GiB22±26.5%
Open_Gpt4_8x7B_v0.2MoEQ4_K_M46.7B26.43 GiB2.25 GiB29.62 GiB0.14 GiB36±37%
GPT-NeoX-20B-ErebusI1-Q3_K_M20.6B10.03 GiB18.56 GiB29.58 GiB0.18 GiB23±26.5%
Skyfall-31B-v4.2Q6_K_L31.4B24.74 GiB3.80 GiB29.55 GiB0.21 GiB23±26.5%
Delphi-25B-SimpleRL-MathI1-IQ3_XS25.0B9.74 GiB18.83 GiB29.54 GiB0.22 GiB23±26.5%
Qwen3.6-27B-Fable-5-ExperimentalQ8_027.8B27.42 GiB1.13 GiB29.51 GiB0.25 GiB23±26.5%
Qwen3-53B-A3B-2507-THINKING-TOTAL-RECALL-v2-MASTER-CODERMoEI1-Q3_K_L53.0B25.64 GiB2.95 GiB29.49 GiB0.27 GiB55±37%
Darwin-35B-A3B-OpusMoEQ6_K_L36.0B28.22 GiB0.35 GiB29.47 GiB0.29 GiB110±37%
Aurora-Code-1MoEQ6_K_L34.7B28.22 GiB0.35 GiB29.47 GiB0.29 GiB110±37%
grug-35b-v2MoEQ6_K_L35.1B28.22 GiB0.35 GiB29.47 GiB0.29 GiB110±37%
grug-35bMoEQ6_K_L35.1B28.22 GiB0.35 GiB29.47 GiB0.29 GiB110±37%
WorldSim-Opus-3.6-35B-A3BMoEQ6_K_L35.1B28.22 GiB0.35 GiB29.47 GiB0.29 GiB110±37%
Qwen3.6-35B-A3B-AnkoMoEQ6_K_L35.1B28.22 GiB0.35 GiB29.47 GiB0.29 GiB110±37%
KAT-Coder-V2.5-DevMoEQ6_K_L34.7B28.22 GiB0.35 GiB29.47 GiB0.29 GiB110±37%
Ornith-1.0-35BMoEQ6_K_L34.7B28.22 GiB0.35 GiB29.47 GiB0.29 GiB110±37%
Nex-N2-miniMoEQ6_K_L35.1B28.22 GiB0.35 GiB29.47 GiB0.29 GiB110±37%
Maenad-70BI1-Q2_K_S70.6B22.79 GiB5.63 GiB29.44 GiB0.32 GiB23±26.5%
DeepSeek-R1-Distill-Llama-70B-Uncensored-v2-Unbiased-ReasonerI1-Q2_K_S70.6B22.79 GiB5.63 GiB29.44 GiB0.32 GiB23±26.5%
Rombos-LLM-70b-Llama-3.3I1-Q2_K_S70.6B22.79 GiB5.63 GiB29.44 GiB0.32 GiB23±26.5%
L3.3-Electra-R1-70bI1-Q2_K_S70.6B22.79 GiB5.63 GiB29.44 GiB0.32 GiB23±26.5%
Latxa-Llama-3.1-70B-Instruct-v2I1-Q2_K_S70.6B22.79 GiB5.63 GiB29.44 GiB0.32 GiB23±26.5%
Llama-3.3_70_b_uncensored_continuedI1-Q2_K_S70.6B22.79 GiB5.63 GiB29.44 GiB0.32 GiB23±26.5%
Llama-3.3-70B-Instruct-abliteratedI1-Q2_K_S70.6B22.79 GiB5.63 GiB29.44 GiB0.32 GiB23±26.5%
grok-oss-Revenant-70BI1-Q2_K_S70.6B22.79 GiB5.63 GiB29.44 GiB0.32 GiB23±26.5%
Llama-3.1-Nemotron-70B-Instruct-HFI1-Q2_K_S70.6B22.79 GiB5.63 GiB29.44 GiB0.32 GiB23±26.5%
L3.3-70B-Euryale-v2.3I1-Q2_K_S70.6B22.79 GiB5.63 GiB29.44 GiB0.32 GiB23±26.5%
Hermes-4-70B-hereticI1-Q2_K_S70.6B22.79 GiB5.63 GiB29.44 GiB0.32 GiB23±26.5%
Llama-3.1-70BQ2_K_S70.6B22.79 GiB5.63 GiB29.44 GiB0.32 GiB23±26.5%
Golem-70B-v1bI1-Q2_K_S70.6B22.79 GiB5.63 GiB29.44 GiB0.32 GiB23±26.5%
DeepSeek-R1-Distill-Llama-70B-abliteratedI1-Q2_K_S70.6B22.79 GiB5.63 GiB29.44 GiB0.32 GiB23±26.5%
DeepSeek-R1-Distill-Llama-70B-hereticI1-Q2_K_S70.6B22.79 GiB5.63 GiB29.44 GiB0.32 GiB23±26.5%
Legion-V2.1-LLaMa-70BI1-Q2_K_S70.6B22.79 GiB5.63 GiB29.44 GiB0.32 GiB23±26.5%
Assistant_Pepe_70BI1-Q2_K_S70.6B22.79 GiB5.63 GiB29.44 GiB0.32 GiB23±26.5%
Athene-70BQ2_K_S70.6B22.79 GiB5.63 GiB29.44 GiB0.32 GiB23±26.5%
Hermes-3-Llama-3.1-70BQ2_K_S70.6B22.79 GiB5.63 GiB29.44 GiB0.32 GiB23±26.5%
L3.3-70B-Magnum-DiamondQ2_K_S70.6B22.79 GiB5.63 GiB29.44 GiB0.32 GiB23±26.5%
Meta-Llama-3-70B-Instruct-abliterated-v3.5Q2_K_S70.6B22.79 GiB5.63 GiB29.44 GiB0.32 GiB23±26.5%
Hypernova-60B-2605MoEI1-IQ3_XXS58.7B27.90 GiB0.57 GiB29.36 GiB0.40 GiB92±37%
DeepSeek-R1-Distill-Llama-70BUD-IQ2_M70.6B22.70 GiB5.63 GiB29.35 GiB0.41 GiB23±26.5%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a Instinct MI60 run?
1995 of 2118 indexed open-weight models fit a Instinct MI60 at 65,536 context with q4_0 KV cache, the largest being Qwen3-Next-80B-A3B-Thinking at Q2_K. That covers text, vision-language, image, video and speech models.
How much usable memory does a Instinct MI60 actually have?
Its nameplate is 32 GB, but about 29.76 GiB is available to a model once driver and compositor overhead is accounted for.
Is a Instinct MI60 fast for local AI?
Its memory bandwidth is 1024 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.