Best local AI models for 20GB VRAM

Ranked by what actually fits at 32K context, computed from real file bytes.

A 20GB card gives you about 18.60 GiB to work with after driver overhead. 1543 indexed models fit at 32K context — the largest being Huihui-Qwen3-Coder-Next-abliterated at 79.7B parameters in I1-IQ1_M.

From the file· fit from summed bytesFrom the file· KV per layer
4GB6GB8GB10GB12GB16GB20GB24GB32GB48GB64GB96GB128GB
alltext generation 1543

Fits in 20GB at 32K context

largest quantization that fits, per model
ModelModalityBest quantParams○Total◐Headroom◐
Qwen3-Coder-30B-A3B-InstructMoEtext generationQ3_K_M30.5B17.50 GiB1.10 GiB
Qwen3.6-27Btext generationIQ4_NL27.8B18.08 GiB0.52 GiB
Qwen3.8-27Btext generationIQ4_NL27.8B18.08 GiB0.52 GiB
gemma-4-12B-it-qat-q4_0-unquantizedtext generationQ4_012.0B9.81 GiB8.79 GiB
gemma-4-E4B-ittext generationBF168.0B15.50 GiB3.10 GiB
Qwen3-30B-A3B-Thinking-2507MoEtext generationQ3_K_M30.5B17.50 GiB1.10 GiB
Qwen3-4Btext generationBF164.0B12.81 GiB5.79 GiB
Qwen3-8Btext generationQ8_08.2B13.44 GiB5.16 GiB
Laguna-XS-2.1MoEtext generationIQ3_M33.4B17.33 GiB1.27 GiB
Llama-3.2-1B-Instructtext generationF161.2B4.11 GiB14.49 GiB
Qwen-AgentWorld-35B-A3BMoEtext generationUD-IQ4_NL34.7B18.30 GiB0.30 GiB
gpt-oss-20bMoEtext generationF1621.5B14.40 GiB4.20 GiB
KAT-Coder-V2.5-DevMoEtext generationUD-IQ4_XS34.7B18.39 GiB0.21 GiB
Qwen3-30B-A3BMoEtext generationQ3_K_M30.5B17.50 GiB1.10 GiB
llama-3-youko-8btext generationQ8_08.0B12.79 GiB5.81 GiB
Llama-3.1-8B-Instructtext generationQ8_08.0B12.79 GiB5.81 GiB
ced-basetext generationF3286M1.16 GiB17.44 GiB
Qwen2.5-7B-Instructtext generationF167.6B16.80 GiB1.80 GiB
UI-TARS-1.5-7Btext generationF168.3B16.80 GiB1.80 GiB
gemma-3-1b-ittext generationF161000M2.81 GiB15.79 GiB
gemma-4-E2B-it-qat-q4_0-unquantizedtext generationBF165.1B9.83 GiB8.77 GiB
Qwen3-1.7Btext generationBF162.0B8.08 GiB10.52 GiB
GLM-4.7-FlashMoEtext generationQ4_K_S31.2B18.54 GiB0.06 GiB
embeddinggemma-300mtext generationF32303M2.05 GiB16.55 GiB
Llama-3.2-3B-Instructtext generationF163.2B10.30 GiB8.30 GiB
Qwen3-0.6Btext generationBF16752M5.68 GiB12.92 GiB
Qwen3-14Btext generationQ6_K14.8B17.15 GiB1.45 GiB
Ornith-1.0-35BMoEtext generationUD-IQ4_NL34.7B18.30 GiB0.30 GiB
Wan2.1-T2V-1.3Btext generationQ8_01.4B12.18 GiB6.42 GiB
LFM2.5-1.2B-Instructtext generationBF161.2B3.37 GiB15.23 GiB
Qwen2.5-Coder-7B-Instructtext generationQ8_07.6B17.69 GiB0.91 GiB
Qwen2.5-32B-Instructtext generationIQ2_S32.8B18.57 GiB0.03 GiB
Qwen2.5-1.5B-Instructtext generationF161.5B4.56 GiB14.04 GiB
Jan-v3-4B-base-instructtext generationBF164.4B13.53 GiB5.07 GiB
gemma-3-4b-ittext generationBF164.3B8.85 GiB9.75 GiB
MiniCPM5-1B-Claude-Opus-Fable5-Thinkingtext generationF161.1B3.55 GiB15.05 GiB
Ornith-1.0-9Btext generationQ8_09.2B10.95 GiB7.65 GiB
gemma-4-26B-A4B-it-ultra-uncensored-hereticMoEtext generationQ4_K_M25.8B17.97 GiB0.63 GiB
Qwen2.5-Coder-32B-Instructtext generationIQ2_S32.8B18.57 GiB0.03 GiB
Qwen2.5-Coder-14B-Instructtext generationQ6_K_L14.8B18.49 GiB0.11 GiB
Qwen3-4B-Instruct-2507text generationF164.0B12.81 GiB5.79 GiB
granite-4.1-3btext generationBF163.4B9.65 GiB8.95 GiB
Qwen2.5-3B-Instructtext generationF323.1B13.44 GiB5.16 GiB
Phi-3.5-mini-instructtext generationQ8_03.8B16.59 GiB2.01 GiB
gemma-2-2b-ittext generationF322.6B12.41 GiB6.19 GiB
Qwen3-30B-A3B-Instruct-2507MoEtext generationQ3_K_M30.5B17.50 GiB1.10 GiB
MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinkingtext generationF161.1B3.55 GiB15.05 GiB
Qwen2.5-0.5B-Instructtext generationF16494M2.08 GiB16.52 GiB
DeepSeek-R1-0528-Qwen3-8Btext generationQ8_08.2B13.44 GiB5.16 GiB
Qwen3-32Btext generationUD-IQ2_XXS32.8B17.53 GiB1.07 GiB
Qwen3-VL-8B-Instructtext generationQ8_08.8B13.44 GiB5.16 GiB
Sugoi-14B-Ultra-HFtext generationI1-Q6_K14.8B18.14 GiB0.46 GiB
gemma-4-12b-heretic-abliteratedtext generationQ8_012.0B15.11 GiB3.49 GiB
TinyLlama-1.1B-Chat-v1.0text generationF161.1B3.53 GiB15.07 GiB
Qwen2.5-14B-Instructtext generationQ6_K_L14.8B18.49 GiB0.11 GiB
Mistral-Nemo-Instruct-2407text generationQ8_012.2B17.98 GiB0.62 GiB
Phi-4-mini-instructtext generationBF163.8B11.96 GiB6.64 GiB
SmolLM2-135M-Instructtext generationF16135M1.72 GiB16.88 GiB
Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16text generationUD-IQ3_S33.0B18.37 GiB0.23 GiB
gemma-3-27b-ittext generationQ4_K_S27.4B18.59 GiB0.01 GiB
○Spec sheet◐Predictedwhat these mean

This page models a generic 20GB accelerator, so it answers what fits rather than how fast it runs. For tokens per second you need a specific card — pick one from hardware, where bandwidth is known.