Best local AI models for 16GB VRAM

Ranked by what actually fits at 32K context, computed from real file bytes.

A 16GB card gives you about 14.88 GiB to work with after driver overhead. 1435 indexed models fit at 32K context — the largest being Qwen3-Coder-Next-REAM at 60.3B parameters in I1-IQ1_M.

From the file· fit from summed bytesFrom the file· KV per layer
4GB6GB8GB10GB12GB16GB20GB24GB32GB48GB64GB96GB128GB
alltext generation 1435

Fits in 16GB at 32K context

largest quantization that fits, per model
ModelModalityBest quantParams○Total◐Headroom◐
Qwen3-Coder-30B-A3B-InstructMoEtext generationQ2_K_L30.5B14.35 GiB0.53 GiB
Qwen3.6-27Btext generationQ3_K_S27.8B14.57 GiB0.31 GiB
Qwen3.8-27Btext generationQ3_K_S27.8B14.57 GiB0.31 GiB
gemma-4-12B-it-qat-q4_0-unquantizedtext generationQ4_012.0B9.81 GiB5.07 GiB
gemma-4-E4B-ittext generationQ8_08.0B8.95 GiB5.93 GiB
Qwen3-30B-A3B-Thinking-2507MoEtext generationQ2_K_L30.5B14.35 GiB0.53 GiB
Qwen3-4Btext generationBF164.0B12.81 GiB2.07 GiB
Qwen3-8Btext generationQ8_08.2B13.44 GiB1.44 GiB
Laguna-XS-2.1MoEtext generationQ2_K_L33.4B13.68 GiB1.20 GiB
Llama-3.2-1B-Instructtext generationF161.2B4.11 GiB10.77 GiB
Qwen-AgentWorld-35B-A3BMoEtext generationUD-IQ3_XXS34.7B14.23 GiB0.65 GiB
gpt-oss-20bMoEtext generationF1621.5B14.40 GiB0.48 GiB
KAT-Coder-V2.5-DevMoEtext generationQ2_K_L34.7B13.64 GiB1.24 GiB
Qwen3-30B-A3BMoEtext generationQ2_K_L30.5B14.35 GiB0.53 GiB
llama-3-youko-8btext generationQ8_08.0B12.79 GiB2.09 GiB
Llama-3.1-8B-Instructtext generationQ8_08.0B12.79 GiB2.09 GiB
ced-basetext generationF3286M1.16 GiB13.72 GiB
Qwen2.5-7B-Instructtext generationQ8_07.6B10.15 GiB4.73 GiB
UI-TARS-1.5-7Btext generationQ8_08.3B10.15 GiB4.73 GiB
gemma-3-1b-ittext generationF161000M2.81 GiB12.07 GiB
gemma-4-E2B-it-qat-q4_0-unquantizedtext generationBF165.1B9.83 GiB5.05 GiB
Qwen3-1.7Btext generationBF162.0B8.08 GiB6.80 GiB
GLM-4.7-FlashMoEtext generationQ3_K_S31.2B14.84 GiB0.04 GiB
embeddinggemma-300mtext generationF32303M2.05 GiB12.83 GiB
Llama-3.2-3B-Instructtext generationF163.2B10.30 GiB4.58 GiB
Qwen3-0.6Btext generationBF16752M5.68 GiB9.20 GiB
Qwen3-14Btext generationQ4_114.8B14.60 GiB0.28 GiB
Ornith-1.0-35BMoEtext generationUD-IQ3_XXS34.7B14.23 GiB0.65 GiB
Wan2.1-T2V-1.3Btext generationQ8_01.4B12.18 GiB2.70 GiB
LFM2.5-1.2B-Instructtext generationBF161.2B3.37 GiB11.51 GiB
Qwen2.5-Coder-7B-Instructtext generationQ6_K7.6B14.25 GiB0.63 GiB
Qwen2.5-1.5B-Instructtext generationF161.5B4.56 GiB10.32 GiB
Jan-v3-4B-base-instructtext generationBF164.4B13.53 GiB1.35 GiB
gemma-3-4b-ittext generationBF164.3B8.85 GiB6.03 GiB
MiniCPM5-1B-Claude-Opus-Fable5-Thinkingtext generationF161.1B3.55 GiB11.33 GiB
Ornith-1.0-9Btext generationQ8_09.2B10.95 GiB3.93 GiB
gemma-4-26B-A4B-it-ultra-uncensored-hereticMoEtext generationQ3_K_M25.8B14.70 GiB0.18 GiB
Qwen2.5-Coder-14B-Instructtext generationQ4_K_S14.8B14.83 GiB0.05 GiB
Qwen3-4B-Instruct-2507text generationF164.0B12.81 GiB2.07 GiB
granite-4.1-3btext generationBF163.4B9.65 GiB5.23 GiB
Qwen2.5-3B-Instructtext generationF323.1B13.44 GiB1.44 GiB
Phi-3.5-mini-instructtext generationQ4_K_S3.8B14.84 GiB0.04 GiB
gemma-2-2b-ittext generationF322.6B12.41 GiB2.47 GiB
Qwen3-30B-A3B-Instruct-2507MoEtext generationQ2_K_L30.5B14.35 GiB0.53 GiB
MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinkingtext generationF161.1B3.55 GiB11.33 GiB
Qwen2.5-0.5B-Instructtext generationF16494M2.08 GiB12.80 GiB
DeepSeek-R1-0528-Qwen3-8Btext generationQ8_08.2B13.44 GiB1.44 GiB
Qwen3-VL-8B-Instructtext generationQ8_08.8B13.44 GiB1.44 GiB
Sugoi-14B-Ultra-HFtext generationI1-Q4_K_S14.8B14.83 GiB0.05 GiB
gemma-4-12b-heretic-abliteratedtext generationI1-Q6_K12.0B12.43 GiB2.45 GiB
TinyLlama-1.1B-Chat-v1.0text generationF161.1B3.53 GiB11.35 GiB
Qwen2.5-14B-Instructtext generationQ4_K_S14.8B14.83 GiB0.05 GiB
Mistral-Nemo-Instruct-2407text generationQ5_K_L12.2B14.36 GiB0.52 GiB
Phi-4-mini-instructtext generationBF163.8B11.96 GiB2.92 GiB
SmolLM2-135M-Instructtext generationF16135M1.72 GiB13.16 GiB
gemma-3-27b-ittext generationIQ3_XS27.4B14.76 GiB0.12 GiB
FastContext-1.0-4B-SFTtext generationF164.0B12.81 GiB2.07 GiB
DeepSeek-R1-Distill-Qwen-7Btext generationQ8_07.6B10.15 GiB4.73 GiB
GLM-4.6V-Flashtext generationQ8_010.3B11.40 GiB3.48 GiB
Qwen2.5-Coder-3B-Instructtext generationF163.1B7.69 GiB7.19 GiB
○Spec sheet◐Predictedwhat these mean

This page models a generic 16GB accelerator, so it answers what fits rather than how fast it runs. For tokens per second you need a specific card — pick one from hardware, where bandwidth is known.