Best local AI models for 10GB VRAM

Ranked by what actually fits at 32K context, computed from real file bytes.

A 10GB card gives you about 9.30 GiB to work with after driver overhead. 1014 indexed models fit at 32K context — the largest being Qwen3.6-35B-A3B-Claude-4.6-Opus-Reasoning-Distilled at 36.0B parameters in I1-IQ1_M.

From the file· fit from summed bytesFrom the file· KV per layer
4GB6GB8GB10GB12GB16GB20GB24GB32GB48GB64GB96GB128GB
alltext generation 1014

Fits in 10GB at 32K context

largest quantization that fits, per model
ModelModalityBest quantParams○Total◐Headroom◐
gemma-4-E4B-ittext generationQ8_08.0B8.95 GiB0.35 GiB
Qwen3-4Btext generationQ8_04.0B9.30 GiB0.00 GiB
Qwen3-8Btext generationQ3_K_M8.2B9.17 GiB0.13 GiB
Llama-3.2-1B-Instructtext generationF161.2B4.11 GiB5.19 GiB
Llama-3.1-8B-Instructtext generationQ4_K_S8.0B9.21 GiB0.09 GiB
ced-basetext generationF3286M1.16 GiB8.14 GiB
Qwen2.5-7B-Instructtext generationQ6_K_L7.6B8.67 GiB0.63 GiB
UI-TARS-1.5-7Btext generationQ6_K8.3B8.43 GiB0.87 GiB
gemma-3-1b-ittext generationF161000M2.81 GiB6.49 GiB
gemma-4-E2B-it-qat-q4_0-unquantizedtext generationQ4_K5.1B4.22 GiB5.08 GiB
Qwen3-1.7Btext generationBF162.0B8.08 GiB1.22 GiB
embeddinggemma-300mtext generationF32303M2.05 GiB7.25 GiB
Llama-3.2-3B-Instructtext generationQ8_03.2B7.50 GiB1.80 GiB
Qwen3-0.6Btext generationBF16752M5.68 GiB3.62 GiB
Wan2.1-T2V-1.3Btext generationQ5_01.4B8.58 GiB0.72 GiB
LFM2.5-1.2B-Instructtext generationBF161.2B3.37 GiB5.93 GiB
Qwen2.5-Coder-7B-Instructtext generationQ6_K_L7.6B8.67 GiB0.63 GiB
Qwen2.5-1.5B-Instructtext generationF161.5B4.56 GiB4.74 GiB
Jan-v3-4B-base-instructtext generationQ6_K_L4.4B8.86 GiB0.44 GiB
gemma-3-4b-ittext generationBF164.3B8.85 GiB0.45 GiB
MiniCPM5-1B-Claude-Opus-Fable5-Thinkingtext generationF161.1B3.55 GiB5.75 GiB
Ornith-1.0-9Btext generationQ6_K9.2B9.01 GiB0.29 GiB
Qwen3-4B-Instruct-2507text generationQ8_04.0B9.30 GiB0.00 GiB
granite-4.1-3btext generationQ8_03.4B6.67 GiB2.63 GiB
Qwen2.5-3B-Instructtext generationF163.1B7.69 GiB1.61 GiB
gemma-2-2b-ittext generationQ8_02.6B5.26 GiB4.04 GiB
MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinkingtext generationF161.1B3.55 GiB5.75 GiB
Qwen2.5-0.5B-Instructtext generationF16494M2.08 GiB7.22 GiB
DeepSeek-R1-0528-Qwen3-8Btext generationQ3_K_M8.2B9.17 GiB0.13 GiB
Qwen3-VL-8B-Instructtext generationQ3_K_M8.8B9.17 GiB0.13 GiB
gemma-4-12b-heretic-abliteratedtext generationI1-Q3_K_M12.0B8.98 GiB0.32 GiB
TinyLlama-1.1B-Chat-v1.0text generationF161.1B3.53 GiB5.77 GiB
Phi-4-mini-instructtext generationQ8_03.8B8.61 GiB0.69 GiB
SmolLM2-135M-Instructtext generationF16135M1.72 GiB7.58 GiB
FastContext-1.0-4B-SFTtext generationQ8_04.0B9.30 GiB0.00 GiB
DeepSeek-R1-Distill-Qwen-7Btext generationQ6_K_L7.6B8.67 GiB0.63 GiB
GLM-4.6V-Flashtext generationQ5_K_L10.3B9.01 GiB0.29 GiB
Qwen2.5-Coder-3B-Instructtext generationF163.1B7.69 GiB1.61 GiB
Mistral-7B-Instruct-v0.3text generationQ4_17.2B9.08 GiB0.22 GiB
Meta-Llama-3-8B-Instructtext generationQ4_K_S8.0B9.21 GiB0.09 GiB
Ternary-Bonsai-8B-unpackedtext generationQ2_08.2B7.36 GiB1.94 GiB
Qwen2.5-Coder-1.5B-Instructtext generationF161.5B4.56 GiB4.74 GiB
Ministral-3-14B-Reasoning-2512text generationUD-IQ1_M13.9B9.27 GiB0.03 GiB
gemma-3-270m-ittext generationF16268M1.38 GiB7.92 GiB
umt5-xxltext generationQ8_05.7B6.48 GiB2.82 GiB
DeepSeek-R1-Distill-Qwen-1.5Btext generationF321.8B8.30 GiB1.00 GiB
LFM2.5-8B-A1BMoEtext generationUD-Q6_K8.5B7.77 GiB1.53 GiB
SmolVLM-500M-Instructtext generationF16507M2.78 GiB6.52 GiB
Llama-3.1-8Btext generationQ4_K_S8.0B9.21 GiB0.09 GiB
tinygemma3_cifartext generationQ8_039M0.93 GiB8.37 GiB
Qwen3.6-14B-A3B-FableVibesMoEtext generationQ3_K_M13.8B7.73 GiB1.57 GiB
Qwen3.6-35B-A3B-Claude-4.6-Opus-Reasoning-DistilledMoEtext generationI1-IQ1_M36.0B9.10 GiB0.20 GiB
Llama3.3-8B-Instruct-Thinking-Heretic-Uncensored-Claude-4.5-Opus-High-Reasoningtext generationI1-Q4_K_S8.0B9.21 GiB0.09 GiB
Qwen3-4B-Thinking-2507text generationQ8_04.0B9.30 GiB0.00 GiB
Ministral-3-3B-Reasoning-2512text generationQ8_04.3B7.46 GiB1.84 GiB
Qwen3-0.6B-Basetext generationF16596M5.39 GiB3.91 GiB
Qwen2-7B-Instructtext generationQ6_K7.6B8.43 GiB0.87 GiB
VibeThinker-3Btext generationBF163.1B7.69 GiB1.61 GiB
Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretictext generationQ6_K9.4B8.78 GiB0.52 GiB
Yi-Coder-9B-Chattext generationQ4_K_L8.8B8.97 GiB0.33 GiB
○Spec sheet◐Predictedwhat these mean

This page models a generic 10GB accelerator, so it answers what fits rather than how fast it runs. For tokens per second you need a specific card — pick one from hardware, where bandwidth is known.