Best local AI models for 10GB VRAM
Ranked by what actually fits at 32K context, computed from real file bytes.
A 10GB card gives you about 9.30 GiB to work with after driver overhead. 100 indexed models fit at 32K context — the largest being Skywork-R1V3-38B at 38.4B parameters in IQ2_XXS.
From the file· fit from summed bytesFrom the file· KV per layer
Fits in 10GB at 32K context
largest quantization that fits, per model
| Model | Modality | Best quant | Params○ | Total◐ | Headroom◐ |
|---|---|---|---|---|---|
| Qwen3.5-9B | vision + language | Q6_K | 9.7B | 8.99 GiB | 0.31 GiB |
| gemma-4-12B-it | vision + language | Q3_K_M | 12.0B | 9.18 GiB | 0.12 GiB |
| Qwen3.5-4B | vision + language | Q8_0 | 4.7B | 6.12 GiB | 3.18 GiB |
| Qwythos-9B-Claude-Mythos-5-1M | vision + language | Q5_K | 9.4B | 8.02 GiB | 1.28 GiB |
| Qwen3.5-0.8B | vision + language | BF16 | 873M | 2.60 GiB | 6.70 GiB |
| gemma-4-E2B-it | vision + language | Q8_0 | 5.1B | 5.74 GiB | 3.56 GiB |
| gemma-4-E4B-it-qat-q4_0-unquantized | vision + language | Q4_0 | 7.9B | 6.12 GiB | 3.18 GiB |
| Qwopus3.6-35B-A3B-v1MoE | vision + language | I1-IQ1_M | 36.0B | 9.10 GiB | 0.20 GiB |
| Qwythos-9B-v2 | vision + language | Q5_K_L | 9.7B | 8.93 GiB | 0.37 GiB |
| Qwen3-VL-4B-Instruct | vision + language | Q8_0 | 4.4B | 9.30 GiB | 0.00 GiB |
| Qwen2.5-VL-7B-Instruct | vision + language | Q6_K_L | 8.3B | 8.67 GiB | 0.63 GiB |
| Qwen3-VL-2B-Instruct | vision + language | BF16 | 2.1B | 7.50 GiB | 1.80 GiB |
| gemma-3-12b-it | vision + language | Q3_K_M | 12.2B | 8.91 GiB | 0.39 GiB |
| Qwen3.5-2B | vision + language | BF16 | 2.3B | 4.80 GiB | 4.50 GiB |
| Qwen3.5-9B | vision + language | Q6_K | 9.7B | 8.88 GiB | 0.42 GiB |
| LFM2.5-VL-1.6B | vision + language | BF16 | 1.6B | 3.37 GiB | 5.93 GiB |
| gemma-4-E4B-it-ultra-uncensored-heretic | vision + language | Q8_0 | 8.0B | 8.80 GiB | 0.50 GiB |
| Unlimited-OCRMoE | vision + language | BF16 | 3.3B | 8.14 GiB | 1.16 GiB |
| Qwopus3.5-9B-v3.5 | vision + language | Q6_K | 9.7B | 8.88 GiB | 0.42 GiB |
| Qwen3.5-9B-GLM5.1-Distill-v1 | vision + language | Q3_K_M | 9.7B | 6.14 GiB | 3.16 GiB |
| Qwable-9B-Claude-Fable-5 | vision + language | I1-Q6_K | 9.4B | 8.69 GiB | 0.61 GiB |
| Qwen3.5-9B-Claude-4.6-OS-Auto-Variable-HERETIC-UNCENSORED-THINKING | vision + language | I1-Q6_K | 9.4B | 8.69 GiB | 0.61 GiB |
| gemma-4-12b-it-uncensored | vision + language | I1-Q3_K_M | 12.0B | 8.98 GiB | 0.32 GiB |
| Holo-3.1-9B | vision + language | I1-Q6_K | 9.4B | 8.69 GiB | 0.61 GiB |
| MiniCPM-V-4_5 | vision + language | Q3_K_M | 8.7B | 9.17 GiB | 0.13 GiB |
| MiniCPM-V-4.6 | vision + language | BF16 | 1.3B | 2.57 GiB | 6.73 GiB |
| Qwen3.5-9B-ultra-uncensored-heretic | vision + language | Q6_K | 9.4B | 8.69 GiB | 0.61 GiB |
| Qwen3-VL-8B-Thinking | vision + language | Q3_K_M | 8.8B | 9.17 GiB | 0.13 GiB |
| Jan-v2-VL-high | vision + language | Q3_K_M | 8.8B | 9.17 GiB | 0.13 GiB |
| LocateAnything-3B | vision + language | BF16 | 3.8B | 8.28 GiB | 1.02 GiB |
| OvisOCR2 | vision + language | F32 | 853M | 3.97 GiB | 5.33 GiB |
| gemma-3n-E2B-it | vision + language | Q8_0 | 5.4B | 5.69 GiB | 3.61 GiB |
| Huihui-Qwen3.5-35B-A3B-abliteratedMoE | vision + language | I1-IQ1_M | 36.0B | 8.94 GiB | 0.36 GiB |
| Qwen2-VL-2B-Instruct | vision + language | F16 | 2.2B | 4.56 GiB | 4.74 GiB |
| Gemma4-12B-Uncensored | vision + language | I1-Q3_K_M | 12.0B | 8.98 GiB | 0.32 GiB |
| Qwen3-VL-Embedding-2B | vision + language | F16 | 2.1B | 7.50 GiB | 1.80 GiB |
| Huihui-Qwen3-VL-4B-Instruct-abliterated | vision + language | Q8_0 | 4.4B | 9.30 GiB | 0.00 GiB |
| medgemma-4b-it | vision + language | BF16 | 4.3B | 8.85 GiB | 0.45 GiB |
| chandra-ocr-2 | vision + language | Q8_0 | 5.3B | 6.62 GiB | 2.68 GiB |
| Qwen3.5-0.8B-Base | vision + language | BF16 | 873M | 2.60 GiB | 6.70 GiB |
| LFM2-VL-1.6B | vision + language | BF16 | 1.6B | 3.37 GiB | 5.93 GiB |
| Qwen2-VL-7B-Instruct | vision + language | Q6_K_L | 8.3B | 8.67 GiB | 0.63 GiB |
| Qwen3-VL-4B-Thinking | vision + language | Q8_0 | 4.4B | 9.30 GiB | 0.00 GiB |
| LFM2.5-VL-450M | vision + language | F32 | 449M | 2.48 GiB | 6.82 GiB |
| Tess-4-9B | vision + language | Q6_K | 9.7B | 9.13 GiB | 0.17 GiB |
| Qwen3.6-9B-Heretic-Uncensored-Thinking-Sweet-Madness | vision + language | Q4_K_M | 9.1B | 6.45 GiB | 2.85 GiB |
| gemma-3n-E4B-it | vision + language | Q8_0 | 7.8B | 8.18 GiB | 1.12 GiB |
| Qwen3-VL-2B-Thinking | vision + language | BF16 | 2.1B | 7.50 GiB | 1.80 GiB |
| Huihui-Qwen3-VL-8B-Instruct-abliterated | vision + language | I1-Q3_K_M | 8.8B | 9.17 GiB | 0.13 GiB |
| Qwen3.5-2B-Base | vision + language | F16 | 2.3B | 4.69 GiB | 4.61 GiB |
| Qwen3.5-4B | vision + language | Q8_0 | 4.7B | 5.99 GiB | 3.31 GiB |
| Qwen3.5-9B-Base | vision + language | I1-Q6_K | 9.7B | 8.69 GiB | 0.61 GiB |
| InternVL3_5-14B | vision + language | Q4_K_M | 15.1B | 9.23 GiB | 0.07 GiB |
| WorldSim-Opus-3.6-35B-A3BMoE | vision + language | I1-IQ1_M | 35.1B | 9.10 GiB | 0.20 GiB |
| gemma-4-E2B-it-qat-q4_0-unquantized-heretic | vision + language | I1-Q4_1 | 5.1B | 4.27 GiB | 5.03 GiB |
| LFM2-VL-450M | vision + language | BF16 | 451M | 1.82 GiB | 7.48 GiB |
| medgemma-1.5-4b-it | vision + language | BF16 | 4.3B | 8.85 GiB | 0.45 GiB |
| MiMo-VL-7B-RL | vision + language | I1-IQ4_XS | 8.3B | 9.30 GiB | 0.00 GiB |
| InternVL3_5-30B-A3B | vision + language | IQ2_S | 30.8B | 8.99 GiB | 0.31 GiB |
| Fara-7B | vision + language | Q6_K_L | 8.3B | 8.67 GiB | 0.63 GiB |
This page models a generic 10GB accelerator, so it answers what fits rather than how fast it runs. For tokens per second you need a specific card — pick one from hardware, where bandwidth is known.