Best local AI models for 48GB VRAM
Ranked by what actually fits at 32K context, computed from real file bytes.
A 48GB card gives you about 44.64 GiB to work with after driver overhead. 184 indexed models fit at 32K context — the largest being XORTRON-NXTXPRTXXL at 128B parameters in I1-IQ2_XXS.
From the file· fit from summed bytesFrom the file· KV per layer
Fits in 48GB at 32K context
largest quantization that fits, per model
| Model | Modality | Best quant | Params○ | Total◐ | Headroom◐ |
|---|---|---|---|---|---|
| Qwen3.6-35B-A3BMoE | vision + language | Q8_0 | 36.0B | 38.50 GiB | 6.14 GiB |
| Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP | vision + language | Q5_K_M | 27.8B | 41.90 GiB | 2.74 GiB |
| Qwen3.5-9B | vision + language | BF16 | 9.7B | 18.98 GiB | 25.66 GiB |
| gemma-4-26B-A4B-itMoE | vision + language | Q8_0 | 26.5B | 28.22 GiB | 16.42 GiB |
| gemma-4-12B-it | vision + language | BF16 | 12.0B | 25.51 GiB | 19.13 GiB |
| Qwen3.5-4B | vision + language | BF16 | 4.7B | 9.88 GiB | 34.76 GiB |
| Qwythos-9B-Claude-Mythos-5-1M | vision + language | BF16 | 9.4B | 35.67 GiB | 8.97 GiB |
| gemma-4-31B-it | vision + language | Q8_0 | 31.3B | 37.93 GiB | 6.71 GiB |
| Muse-Glimmer-30B | vision + language | Q8_0 | 29.8B | 31.51 GiB | 13.13 GiB |
| gemma-4-26B-A4B-it-qat-q4_0-unquantizedMoE | vision + language | Q4_0 | 26.5B | 15.78 GiB | 28.86 GiB |
| Qwen3.5-122B-A10BMoE | vision + language | IQ2_M | 125B | 42.97 GiB | 1.67 GiB |
| Qwen3.5-0.8B | vision + language | BF16 | 873M | 2.60 GiB | 42.04 GiB |
| gemma-4-E2B-it | vision + language | BF16 | 5.1B | 9.71 GiB | 34.93 GiB |
| gemma-4-31B-it-qat-q4_0-unquantized | vision + language | Q4_0 | 32.7B | 23.49 GiB | 21.15 GiB |
| gemma-4-E4B-it-qat-q4_0-unquantized | vision + language | Q4_0 | 7.9B | 6.12 GiB | 38.52 GiB |
| Qwen3-VL-30B-A3B-InstructMoE | vision + language | Q8_0 | 31.1B | 34.05 GiB | 10.59 GiB |
| Qwopus3.6-35B-A3B-v1MoE | vision + language | Q8_0 | 36.0B | 36.64 GiB | 8.00 GiB |
| Qwen3.5-9B-The-Defiant-Fable-Uncensored-Heretic-NEO-IMATRIX-MAX-MTP | vision + language | Q8_0 | 9.7B | 21.60 GiB | 23.04 GiB |
| Qwen3.5-35B-A3BMoE | vision + language | Q8_0 | 36.0B | 36.65 GiB | 7.99 GiB |
| Qwythos-9B-v2 | vision + language | BF16 | 9.7B | 35.67 GiB | 8.97 GiB |
| ThinkingCap-Qwen3.6-27B | vision + language | Q8_0 | 27.4B | 29.98 GiB | 14.66 GiB |
| Qwopus3.6-27B-Coder | vision + language | Q8_0 | 27.8B | 29.92 GiB | 14.72 GiB |
| Qwen3-VL-4B-Instruct | vision + language | BF16 | 4.4B | 12.81 GiB | 31.83 GiB |
| Qwen3.6-40B-Claude-4.6-Opus-Deckard-Heretic-Uncensored-Thinking | vision + language | Q8_0 | 39.5B | 43.77 GiB | 0.87 GiB |
| Qwen2.5-VL-7B-Instruct | vision + language | BF16 | 8.3B | 16.80 GiB | 27.84 GiB |
| Qwen3-VL-2B-Instruct | vision + language | BF16 | 2.1B | 7.50 GiB | 37.14 GiB |
| Qwen3.5-27B | vision + language | Q8_0 | 27.8B | 29.98 GiB | 14.66 GiB |
| gemma-3-12b-it | vision + language | BF16 | 12.2B | 25.24 GiB | 19.40 GiB |
| Qwen3.5-2B | vision + language | BF16 | 2.3B | 4.80 GiB | 39.84 GiB |
| Qwen3.6-27B-Heretic2-Uncensored-Finetune-Thinking | vision + language | Q8_0 | 27.4B | 30.68 GiB | 13.96 GiB |
| Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-DistilledMoE | vision + language | Q8_0 | 36.0B | 36.63 GiB | 8.01 GiB |
| Qwen3.6-35B-A3B-uncensored-heretic-Native-MTP-PreservedMoE | vision + language | Q8_0 | 35.1B | 36.64 GiB | 8.00 GiB |
| Qwen3.5-9B | vision + language | Q8_0 | 9.7B | 10.95 GiB | 33.69 GiB |
| Qwen3.6-35B-A3B-uncensored-hereticMoE | vision + language | Q8_0 | 35.1B | 35.80 GiB | 8.84 GiB |
| gemma-4-31B-it-uncensored-heretic | vision + language | Q8_0 | 31.3B | 37.45 GiB | 7.19 GiB |
| Qwopus3.6-27B-v2 | vision + language | Q8_0 | 27.8B | 29.92 GiB | 14.72 GiB |
| Qwen3.6-27B-uncensored-heretic-v2-Native-MTP-Preserved | vision + language | Q8_0 | 27.4B | 37.66 GiB | 6.98 GiB |
| Qwopus3.5-122B-A10B-Kimi-K2.6-destill-healed-abliteratedMoE | vision + language | Q2_K | 123B | 44.24 GiB | 0.40 GiB |
| Mistral-Small-3.2-24B-Instruct-2506 | vision + language | Q8_0 | 24.0B | 29.25 GiB | 15.39 GiB |
| LFM2.5-VL-1.6B | vision + language | BF16 | 1.6B | 3.37 GiB | 41.27 GiB |
| gemma-4-E4B-it-ultra-uncensored-heretic | vision + language | BF16 | 8.0B | 15.34 GiB | 29.30 GiB |
| Unlimited-OCRMoE | vision + language | BF16 | 3.3B | 8.14 GiB | 36.50 GiB |
| diffusiongemma-26B-A4B-itMoE | vision + language | Q8_0 | 25.8B | 27.36 GiB | 17.28 GiB |
| Qwopus3.5-9B-v3.5 | vision + language | BF16 | 9.7B | 18.53 GiB | 26.11 GiB |
| Qwen3.6-27B-uncensored-heretic-v2 | vision + language | Q8_0 | 27.4B | 36.82 GiB | 7.82 GiB |
| Tess-4-27B | vision + language | Q8_0 | 27.8B | 32.44 GiB | 12.20 GiB |
| Qwen3.5-9B-GLM5.1-Distill-v1 | vision + language | Q8_0 | 9.7B | 17.56 GiB | 27.08 GiB |
| Qwable-9B-Claude-Fable-5 | vision + language | F16 | 9.4B | 18.53 GiB | 26.11 GiB |
| Llama-4-Scout-17B-16E-InstructMoE | vision + language | UD-IQ2_M | 109B | 43.22 GiB | 1.42 GiB |
| Qwen3-VL-32B-Instruct | vision + language | Q8_0 | 33.4B | 41.32 GiB | 3.32 GiB |
| UI-TARS-72B-DPO | vision + language | IQ3_M | 73.4B | 44.00 GiB | 0.64 GiB |
| Qwen3.5-9B-Claude-4.6-OS-Auto-Variable-HERETIC-UNCENSORED-THINKING | vision + language | BF16 | 9.4B | 18.53 GiB | 26.11 GiB |
| Qwen3.6-35B-A3BMoE | vision + language | Q8_0 | 36.0B | 36.63 GiB | 8.01 GiB |
| gemma-4-12b-it-uncensored | vision + language | F16 | 12.0B | 25.51 GiB | 19.13 GiB |
| gemma-4-26B-A4B-it-qat-q4_0-unquantized-uncensored-hereticMoE | vision + language | NVFP4 | 25.8B | 18.79 GiB | 25.85 GiB |
| Holo-3.1-9B | vision + language | F16 | 9.4B | 18.53 GiB | 26.11 GiB |
| MiniCPM-V-4_5 | vision + language | F16 | 8.7B | 35.85 GiB | 8.79 GiB |
| MiniCPM-V-4.6 | vision + language | BF16 | 1.3B | 2.57 GiB | 42.07 GiB |
| Ornith-Agents-A1-3.6-35B-A3B-dare_tiesMoE | vision + language | Q8_0 | 34.7B | 37.03 GiB | 7.61 GiB |
| Qwen3.5-9B-ultra-uncensored-heretic | vision + language | F16 | 9.4B | 18.53 GiB | 26.11 GiB |
This page models a generic 48GB accelerator, so it answers what fits rather than how fast it runs. For tokens per second you need a specific card — pick one from hardware, where bandwidth is known.