Best local AI models for 96GB VRAM

Ranked by what actually fits at 32K context, computed from real file bytes.

A 96GB card gives you about 89.28 GiB to work with after driver overhead. 1788 indexed models fit at 32K context — the largest being Trinity-Large-Thinking at 399B parameters in IQ1_M.

From the file· fit from summed bytesFrom the file· KV per layer
4GB6GB8GB10GB12GB16GB20GB24GB32GB48GB64GB96GB128GB
alltext generation 1788

Fits in 96GB at 32K context

largest quantization that fits, per model
ModelModalityBest quantParams○Total◐Headroom◐
Qwen3-Coder-30B-A3B-InstructMoEtext generationBF1630.5B60.69 GiB28.59 GiB
Qwen3.6-27Btext generationBF1627.8B53.77 GiB35.51 GiB
Qwen3.8-27Btext generationBF1627.8B58.51 GiB30.77 GiB
DeepSeek-V4-FlashMoEtext generationUD-IQ2_M291B85.59 GiB3.69 GiB
gemma-4-12B-it-qat-q4_0-unquantizedtext generationQ4_012.0B9.81 GiB79.47 GiB
gemma-4-E4B-ittext generationBF168.0B15.50 GiB73.78 GiB
Qwen3-30B-A3B-Thinking-2507MoEtext generationBF1630.5B60.69 GiB28.59 GiB
Qwen3-4Btext generationBF164.0B12.81 GiB76.47 GiB
Qwen3-8Btext generationBF168.2B20.60 GiB68.68 GiB
Laguna-XS-2.1MoEtext generationBF1633.4B64.50 GiB24.78 GiB
DeepSeek-V4-Flash-0731MoEtext generationUD-IQ2_M304B85.59 GiB3.69 GiB
Llama-3.2-1B-Instructtext generationF161.2B4.11 GiB85.17 GiB
Qwen-AgentWorld-35B-A3BMoEtext generationBF1634.7B67.62 GiB21.66 GiB
gpt-oss-20bMoEtext generationF1621.5B14.40 GiB74.88 GiB
KAT-Coder-V2.5-DevMoEtext generationBF1634.7B66.04 GiB23.24 GiB
Qwen3-30B-A3BMoEtext generationBF1630.5B60.69 GiB28.59 GiB
llama-3-youko-8btext generationQ8_08.0B12.79 GiB76.49 GiB
Laguna-S-2.1MoEtext generationUD-Q5_K_M118B84.29 GiB4.99 GiB
Llama-3.1-8B-Instructtext generationF328.0B34.76 GiB54.52 GiB
ced-basetext generationF3286M1.16 GiB88.12 GiB
Hy3MoEtext generationIQ2_XXS299B87.31 GiB1.97 GiB
Qwen2.5-7B-Instructtext generationF167.6B16.80 GiB72.48 GiB
UI-TARS-1.5-7Btext generationF168.3B16.80 GiB72.48 GiB
gemma-3-1b-ittext generationF161000M2.81 GiB86.47 GiB
gemma-4-E2B-it-qat-q4_0-unquantizedtext generationBF165.1B9.83 GiB79.45 GiB
Qwen3-1.7Btext generationBF162.0B8.08 GiB81.20 GiB
GLM-4.7-FlashMoEtext generationBF1631.2B58.26 GiB31.02 GiB
embeddinggemma-300mtext generationF32303M2.05 GiB87.23 GiB
Llama-3.2-3B-Instructtext generationF163.2B10.30 GiB78.98 GiB
Qwen3-0.6Btext generationBF16752M5.68 GiB83.60 GiB
Qwen3-14Btext generationBF1614.8B33.37 GiB55.91 GiB
Ornith-1.0-35BMoEtext generationBF1634.7B67.62 GiB21.66 GiB
Wan2.1-T2V-1.3Btext generationQ4_K_M1.4B16.47 GiB72.81 GiB
Qwen3-Coder-NextMoEtext generationQ8_079.7B82.78 GiB6.50 GiB
LFM2.5-1.2B-Instructtext generationBF161.2B3.37 GiB85.91 GiB
Qwen2.5-Coder-7B-Instructtext generationQ8_07.6B17.69 GiB71.59 GiB
Qwen2.5-32B-Instructtext generationF1632.8B69.93 GiB19.35 GiB
Qwen2.5-1.5B-Instructtext generationF161.5B4.56 GiB84.72 GiB
Jan-v3-4B-base-instructtext generationBF164.4B13.53 GiB75.75 GiB
gemma-3-4b-ittext generationBF164.3B8.85 GiB80.43 GiB
MiniCPM5-1B-Claude-Opus-Fable5-Thinkingtext generationF161.1B3.55 GiB85.73 GiB
Ornith-1.0-9Btext generationBF169.2B18.98 GiB70.30 GiB
Agents-A1MoEtext generationF1635.1B66.04 GiB23.24 GiB
gemma-4-26B-A4B-it-ultra-uncensored-hereticMoEtext generationBF1625.8B49.37 GiB39.91 GiB
Qwen2.5-Coder-32B-Instructtext generationQ8_032.8B73.76 GiB15.52 GiB
Qwen2.5-Coder-14B-Instructtext generationQ8_014.8B36.09 GiB53.19 GiB
Qwen3-4B-Instruct-2507text generationF164.0B12.81 GiB76.47 GiB
granite-4.1-3btext generationBF163.4B9.65 GiB79.63 GiB
Qwen2.5-3B-Instructtext generationF323.1B13.44 GiB75.84 GiB
Phi-3.5-mini-instructtext generationF323.8B27.04 GiB62.24 GiB
gemma-2-2b-ittext generationF322.6B12.41 GiB76.87 GiB
Qwen3-30B-A3B-Instruct-2507MoEtext generationBF1630.5B60.69 GiB28.59 GiB
MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinkingtext generationF161.1B3.55 GiB85.73 GiB
Qwen2.5-0.5B-Instructtext generationF16494M2.08 GiB87.20 GiB
DeepSeek-R1-0528-Qwen3-8Btext generationBF168.2B20.60 GiB68.68 GiB
Qwen3-32Btext generationBF1632.8B69.92 GiB19.36 GiB
Qwen3-VL-8B-Instructtext generationBF168.8B20.60 GiB68.68 GiB
Sugoi-14B-Ultra-HFtext generationF1614.8B34.36 GiB54.92 GiB
gemma-4-12b-heretic-abliteratedtext generationQ8_012.0B15.11 GiB74.17 GiB
Qwen3-235B-A22BMoEtext generationQ2_K_L235B86.65 GiB2.63 GiB
○Spec sheet◐Predictedwhat these mean

This page models a generic 96GB accelerator, so it answers what fits rather than how fast it runs. For tokens per second you need a specific card — pick one from hardware, where bandwidth is known.