AMD · workstation

Radeon AI Pro R9700

Radeon AI Pro R9700 has 32 GB of VRAM at 640 GB/s — about 29.76 GiB usable after driver and compositor overhead. 1949 of 2118 indexed models fit at 64K context with q8_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
32 GB
GDDR6
Bandwidth
640 GB/s
256-bit bus
Tensor FP16
—
dense
TDP
300 W
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 1668image 2vision language 177video 16embedding 26audio tts 21audio asr 39

What fits at 64K context

largest quantization that fits, per model · 1949 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
Hermes-4-70BUD-IQ2_XXS70.6B18.10 GiB10.63 GiB29.75 GiB0.01 GiB14±26.5%
Llama-3.3-70B-InstructUD-IQ2_XXS70.6B18.10 GiB10.63 GiB29.75 GiB0.01 GiB14±26.5%
Janus-Pro-7BF167.4B12.88 GiB15.94 GiB29.74 GiB0.02 GiB14±26.5%
Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTPIQ3_M27.8B26.65 GiB2.13 GiB29.74 GiB0.02 GiB14±26.5%
Qwen3.5-27B-HERETIC-Polaris-Advanced-Thinking-Alpha-uncensoredQ8_027.4B26.63 GiB2.13 GiB29.72 GiB0.04 GiB14±26.5%
Qwen3.5-27B-Deckard-PKD-Heretic-Uncensored-ThinkingQ8_027.4B26.63 GiB2.13 GiB29.72 GiB0.04 GiB14±26.5%
Qwen3.6-27B-Heretic2-ThinkingQ8_027.4B26.63 GiB2.13 GiB29.72 GiB0.04 GiB14±26.5%
Qwen3.6-27B-abliteratedQ8_027.4B26.63 GiB2.13 GiB29.72 GiB0.04 GiB14±26.5%
Webcoda-AI-27BQ8_027.4B26.63 GiB2.13 GiB29.72 GiB0.04 GiB14±26.5%
KoQweopus-3.5-27B-experimentalQ8_027.8B26.63 GiB2.13 GiB29.72 GiB0.04 GiB14±26.5%
Huihui-Qwen3.6-27B-abliteratedQ8_027.8B26.63 GiB2.13 GiB29.72 GiB0.04 GiB14±26.5%
Qwen3.5-27B-hereticQ8_027.4B26.63 GiB2.13 GiB29.72 GiB0.04 GiB14±26.5%
Huihui-Qwen3.5-27B-abliteratedQ8_027.8B26.63 GiB2.13 GiB29.72 GiB0.04 GiB14±26.5%
Qwen3.5-Queen-27BQ8_027.4B26.63 GiB2.13 GiB29.72 GiB0.04 GiB14±26.5%
Qwen3.5-27B-abliteratedQ8_026.9B26.63 GiB2.13 GiB29.72 GiB0.04 GiB14±26.5%
ThinkingCap-Qwen3.6-27B-hereticQ8_027.4B26.63 GiB2.13 GiB29.72 GiB0.04 GiB14±26.5%
MusaCoder-27BQ8_0—26.63 GiB2.13 GiB29.72 GiB0.04 GiB14±26.5%
Qwen-Image-BenchQ8_027.4B26.63 GiB2.13 GiB29.72 GiB0.04 GiB14±26.5%
Qwen3.5-27B-uncensored-heretic-v1Q8_027.4B26.63 GiB2.13 GiB29.72 GiB0.04 GiB14±26.5%
Bonsai-27B-unpackedQ8_027.4B26.63 GiB2.13 GiB29.72 GiB0.04 GiB14±26.5%
Ternary-Bonsai-27B-unpackedQ8_027.4B26.63 GiB2.13 GiB29.72 GiB0.04 GiB14±26.5%
Qwen3.6-27B-Claude-Opus-Reasoning-Distill-v2-hereticQ8_027.4B26.63 GiB2.13 GiB29.72 GiB0.04 GiB14±26.5%
Darwin-28B-REASONQ8_026.9B26.63 GiB2.13 GiB29.72 GiB0.04 GiB14±26.5%
Huihui-Qwen3.5-27B-Claude-4.6-Opus-abliteratedQ8_027.8B26.63 GiB2.13 GiB29.72 GiB0.04 GiB14±26.5%
Qwen3.5-27B-Claude-4.6-Opus-Reasoning-DistilledQ8_027.8B26.63 GiB2.13 GiB29.72 GiB0.04 GiB14±26.5%
Qwen3.5-27BQ8_027.8B26.63 GiB2.13 GiB29.72 GiB0.04 GiB14±26.5%
Qwen3.5-27B-WebNovel-Writer-zhQ8_026.9B26.63 GiB2.13 GiB29.72 GiB0.04 GiB14±26.5%
Assistant_Pepe_70BIQ1_S70.6B18.07 GiB10.63 GiB29.72 GiB0.04 GiB14±26.5%
granite-4.1-30bQ5_128.9B20.19 GiB8.50 GiB29.70 GiB0.06 GiB14±26.5%
Gemma-4-Novelist-Eclipse-31BQ5_K_L32.7B22.77 GiB5.94 GiB29.68 GiB0.08 GiB14±26.5%
Gemma-4-31B-StyleTuneQ5_K_L32.7B22.77 GiB5.94 GiB29.68 GiB0.08 GiB14±26.5%
Devstral-Small-2-24B-Instruct-2512Q8_024.0B23.33 GiB5.31 GiB29.67 GiB0.09 GiB14±26.5%
Magistral-Small-2509Q8_024.0B23.33 GiB5.31 GiB29.67 GiB0.09 GiB14±26.5%
Magistral-Small-2507Q8_023.6B23.33 GiB5.31 GiB29.67 GiB0.09 GiB14±26.5%
Transformed-Journey-24BQ8_023.6B23.33 GiB5.31 GiB29.66 GiB0.10 GiB14±26.5%
Magistry-24B-v1.1Q8_023.6B23.33 GiB5.31 GiB29.66 GiB0.10 GiB14±26.5%
Mergedonia-AETHER-24B-v1aQ8_023.6B23.33 GiB5.31 GiB29.66 GiB0.10 GiB14±26.5%
Mergedonia-AETHER-24B-v1bQ8_023.6B23.33 GiB5.31 GiB29.66 GiB0.10 GiB14±26.5%
Slimaki-Tavern-24B-v1.3Q8_023.6B23.33 GiB5.31 GiB29.66 GiB0.10 GiB14±26.5%
Maginum-Cydoms-24BQ8_023.6B23.33 GiB5.31 GiB29.66 GiB0.10 GiB14±26.5%
Maginum-Cydoms-24B-absolute-heresyQ8_023.6B23.33 GiB5.31 GiB29.66 GiB0.10 GiB14±26.5%
Dolphin3.0-Mistral-24BQ8_023.6B23.33 GiB5.31 GiB29.66 GiB0.10 GiB14±26.5%
Dolphin3.0-R1-Mistral-24BQ8_023.6B23.33 GiB5.31 GiB29.66 GiB0.10 GiB14±26.5%
Cydonia_VistralQ8_023.6B23.33 GiB5.31 GiB29.66 GiB0.10 GiB14±26.5%
Mistral-Small-3.2-24B-Instruct-2506-ultra-uncensored-hereticQ8_024.0B23.33 GiB5.31 GiB29.66 GiB0.10 GiB14±26.5%
Huihui-Mistral-Small-3.2-24B-Instruct-2506-abliterated-llamacppfixedQ8_024.0B23.33 GiB5.31 GiB29.66 GiB0.10 GiB14±26.5%
Dans-PersonalityEngine-V1.2.0-24bQ8_023.6B23.33 GiB5.31 GiB29.66 GiB0.10 GiB14±26.5%
Mistral-Small-3_2-24B-Instruct-2506-antislop.v2Q8_024.0B23.33 GiB5.31 GiB29.66 GiB0.10 GiB14±26.5%
Mistral-Small-3.2-24B-Instruct-2506Q8_024.0B23.33 GiB5.31 GiB29.66 GiB0.10 GiB14±26.5%
Dans-PersonalityEngine-V1.3.0-24bQ8_023.6B23.33 GiB5.31 GiB29.66 GiB0.10 GiB14±26.5%
Devstral-Small-2507Q8_023.6B23.33 GiB5.31 GiB29.66 GiB0.10 GiB14±26.5%
Goetia-24B-v1.1Q8_023.6B23.33 GiB5.31 GiB29.66 GiB0.10 GiB14±26.5%
Devstral-Small-2505Q8_023.6B23.33 GiB5.31 GiB29.66 GiB0.10 GiB14±26.5%
MS3.2-PaintedFantasy-v3-24BQ8_023.6B23.33 GiB5.31 GiB29.66 GiB0.10 GiB14±26.5%
MagiSeek-Pro-V1Q8_023.6B23.33 GiB5.31 GiB29.66 GiB0.10 GiB14±26.5%
Precog-24B-v1Q8_0—23.33 GiB5.31 GiB29.66 GiB0.10 GiB14±26.5%
Cogidonia-v2-24BQ8_023.6B23.33 GiB5.31 GiB29.66 GiB0.10 GiB14±26.5%
Magidonia-24B-v4.3-heretic-v1.2Q8_023.6B23.33 GiB5.31 GiB29.66 GiB0.10 GiB14±26.5%
Magidonia-24B-v4.3Q8_0—23.33 GiB5.31 GiB29.66 GiB0.10 GiB14±26.5%
experiment024bQ8_023.6B23.33 GiB5.31 GiB29.66 GiB0.10 GiB14±26.5%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a Radeon AI Pro R9700 run?
1949 of 2118 indexed open-weight models fit a Radeon AI Pro R9700 at 65,536 context with q8_0 KV cache, the largest being Hermes-4-70B at UD-IQ2_XXS. That covers text, vision-language, image, video and speech models.
How much usable memory does a Radeon AI Pro R9700 actually have?
Its nameplate is 32 GB, but about 29.76 GiB is available to a model once driver and compositor overhead is accounted for.
Is a Radeon AI Pro R9700 fast for local AI?
Its memory bandwidth is 640 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.