Best local AI models for 6GB VRAM
Ranked by what actually fits at 32K context, computed from real file bytes.
A 6GB card gives you about 5.58 GiB to work with after driver overhead. 67 indexed models fit at 32K context — the largest being InternVL3_5-14B at 15.1B parameters in IQ2_S.
From the file· fit from summed bytesFrom the file· KV per layer
Fits in 6GB at 32K context
largest quantization that fits, per model
| Model | Modality | Best quant | Params○ | Total◐ | Headroom◐ |
|---|---|---|---|---|---|
| Qwen3.5-9B | vision + language | UD-IQ2_M | 9.7B | 5.53 GiB | 0.05 GiB |
| Qwen3.5-4B | vision + language | Q6_K_L | 4.7B | 5.50 GiB | 0.08 GiB |
| Qwen3.5-0.8B | vision + language | BF16 | 873M | 2.60 GiB | 2.98 GiB |
| gemma-4-E2B-it | vision + language | Q6_K_L | 5.1B | 5.29 GiB | 0.29 GiB |
| Qwythos-9B-v2 | vision + language | I1-Q2_K | 9.7B | 5.48 GiB | 0.10 GiB |
| Qwen2.5-VL-7B-Instruct | vision + language | UD-IQ3_XXS | 8.3B | 5.55 GiB | 0.03 GiB |
| Qwen3-VL-2B-Instruct | vision + language | Q5_K_L | 2.1B | 5.54 GiB | 0.04 GiB |
| Qwen3.5-2B | vision + language | BF16 | 2.3B | 4.80 GiB | 0.78 GiB |
| LFM2.5-VL-1.6B | vision + language | BF16 | 1.6B | 3.37 GiB | 2.21 GiB |
| gemma-4-E4B-it-ultra-uncensored-heretic | vision + language | Q2_K | 8.0B | 5.42 GiB | 0.16 GiB |
| Unlimited-OCRMoE | vision + language | Q8_0 | 3.3B | 5.58 GiB | 0.00 GiB |
| Qwopus3.5-9B-v3.5 | vision + language | Q2_K | 9.7B | 5.40 GiB | 0.18 GiB |
| Qwable-9B-Claude-Fable-5 | vision + language | I1-IQ3_XXS | 9.4B | 5.50 GiB | 0.08 GiB |
| Qwen3.5-9B-Claude-4.6-OS-Auto-Variable-HERETIC-UNCENSORED-THINKING | vision + language | I1-IQ3_XXS | 9.4B | 5.37 GiB | 0.21 GiB |
| Holo-3.1-9B | vision + language | I1-IQ3_XXS | 9.4B | 5.50 GiB | 0.08 GiB |
| MiniCPM-V-4.6 | vision + language | BF16 | 1.3B | 2.57 GiB | 3.01 GiB |
| Qwen3.5-9B-ultra-uncensored-heretic | vision + language | Q2_K | 9.4B | 5.40 GiB | 0.18 GiB |
| LocateAnything-3B | vision + language | Q5_K_M | 3.8B | 4.21 GiB | 1.37 GiB |
| OvisOCR2 | vision + language | F32 | 853M | 3.97 GiB | 1.61 GiB |
| gemma-3n-E2B-it | vision + language | Q6_K_L | 5.4B | 5.27 GiB | 0.31 GiB |
| Qwen2-VL-2B-Instruct | vision + language | F16 | 2.2B | 4.56 GiB | 1.02 GiB |
| Qwen3-VL-Embedding-2B | vision + language | Q5_K_M | 2.1B | 5.47 GiB | 0.11 GiB |
| medgemma-4b-it | vision + language | Q8_0 | 4.3B | 5.46 GiB | 0.12 GiB |
| chandra-ocr-2 | vision + language | Q6_K | 5.3B | 5.52 GiB | 0.06 GiB |
| Qwen3.5-0.8B-Base | vision + language | BF16 | 873M | 2.60 GiB | 2.98 GiB |
| LFM2-VL-1.6B | vision + language | BF16 | 1.6B | 3.37 GiB | 2.21 GiB |
| Qwen2-VL-7B-Instruct | vision + language | Q2_K | 8.3B | 5.41 GiB | 0.17 GiB |
| LFM2.5-VL-450M | vision + language | F32 | 449M | 2.48 GiB | 3.10 GiB |
| Tess-4-9B | vision + language | I1-Q2_K | 9.7B | 5.48 GiB | 0.10 GiB |
| gemma-3n-E4B-it | vision + language | Q4_K_M | 7.8B | 5.56 GiB | 0.02 GiB |
| Qwen3-VL-2B-Thinking | vision + language | Q5_K_M | 2.1B | 5.47 GiB | 0.11 GiB |
| Qwen3.5-2B-Base | vision + language | F16 | 2.3B | 4.69 GiB | 0.89 GiB |
| Qwen3.5-4B | vision + language | Q6_K | 4.7B | 5.04 GiB | 0.54 GiB |
| Qwen3.5-9B-Base | vision + language | I1-IQ3_XXS | 9.7B | 5.50 GiB | 0.08 GiB |
| InternVL3_5-14B | vision + language | IQ2_S | 15.1B | 5.47 GiB | 0.11 GiB |
| gemma-4-E2B-it-qat-q4_0-unquantized-heretic | vision + language | I1-Q4_1 | 5.1B | 4.27 GiB | 1.31 GiB |
| LFM2-VL-450M | vision + language | BF16 | 451M | 1.82 GiB | 3.76 GiB |
| medgemma-1.5-4b-it | vision + language | Q8_0 | 4.3B | 5.46 GiB | 0.12 GiB |
| Fara-7B | vision + language | IQ3_XXS | 8.3B | 5.50 GiB | 0.08 GiB |
| ToriiGate-0.5 | vision + language | Q6_K | 5.2B | 5.52 GiB | 0.06 GiB |
| LFM2-VL-3B | vision + language | Q8_0 | 3.0B | 3.85 GiB | 1.73 GiB |
| llava-llama-3-8b-v1_1-transformers | vision + language | Q4_K_M | 8.4B | 5.42 GiB | 0.16 GiB |
| Qwen2-VL-7B-Instruct-abliterated | vision + language | Q2_K | 8.3B | 5.41 GiB | 0.17 GiB |
| Gemma-3-4B-VL-it-Gemini-Pro-Heretic-Uncensored-Thinking | vision + language | Q8_0 | 4.3B | 5.46 GiB | 0.12 GiB |
| GLM-4.1V-9B-Thinking | vision + language | UD-IQ2_XXS | 10.3B | 5.36 GiB | 0.22 GiB |
| Fara1.5-9B | vision + language | IQ2_M | 9.4B | 5.35 GiB | 0.23 GiB |
| Qwen3.5-0.8B-heretic | vision + language | Q8_0 | 853M | 2.17 GiB | 3.41 GiB |
| Qwen3.5-4B-heretic | vision + language | Q5_K_M | 4.5B | 4.68 GiB | 0.90 GiB |
| NuExtract3 | vision + language | Q6_K | 4.5B | 5.04 GiB | 0.54 GiB |
| Qwen3.5-2B-heretic | vision + language | Q8_0 | 2.2B | 3.56 GiB | 2.02 GiB |
| InternVL3_5-8B | vision + language | Q4_K_M | 8.5B | 5.53 GiB | 0.05 GiB |
| glm-4v-9b | vision + language | Q3_K_M | 13.9B | 5.48 GiB | 0.10 GiB |
| grug-9b | vision + language | IQ2_M | 9.4B | 5.35 GiB | 0.23 GiB |
| Nanonets-OCR-s | vision + language | Q8_0 | 3.8B | 5.00 GiB | 0.58 GiB |
| gemma-3-4b-it-abliterated | vision + language | Q8_0 | 4.3B | 5.46 GiB | 0.12 GiB |
| UI-TARS-7B-DPO | vision + language | Q2_K | 8.3B | 5.41 GiB | 0.17 GiB |
| lift | vision + language | Q2_K | 9.7B | 5.48 GiB | 0.10 GiB |
| MiniCPM-V-4.6-Thinking | vision + language | F16 | 1.3B | 2.57 GiB | 3.01 GiB |
| AfriqueQwen3.5-4B | vision + language | I1-Q6_K | 5.2B | 5.52 GiB | 0.06 GiB |
| MiniCPM-V-4 | vision + language | Q8_0 | 4.1B | 5.38 GiB | 0.20 GiB |
This page models a generic 6GB accelerator, so it answers what fits rather than how fast it runs. For tokens per second you need a specific card — pick one from hardware, where bandwidth is known.