Best local AI models for 48GB VRAM
Ranked by what actually fits at 32K context, computed from real file bytes.
A 48GB card gives you about 44.64 GiB to work with after driver overhead. 1737 indexed models fit at 32K context — the largest being Mixtral-8x22B-Instruct-v0.1 at 141B parameters in IQ1_M.
From the file· fit from summed bytesFrom the file· KV per layer
Fits in 48GB at 32K context
largest quantization that fits, per model
| Model | Modality | Best quant | Params○ | Total◐ | Headroom◐ |
|---|---|---|---|---|---|
| Qwen3-Coder-30B-A3B-InstructMoE | text generation | Q8_0 | 30.5B | 34.05 GiB | 10.59 GiB |
| Qwen3.6-27B | text generation | Q8_0 | 27.8B | 29.92 GiB | 14.72 GiB |
| Qwen3.8-27B | text generation | Q8_0 | 27.8B | 32.44 GiB | 12.20 GiB |
| gemma-4-12B-it-qat-q4_0-unquantized | text generation | Q4_0 | 12.0B | 9.81 GiB | 34.83 GiB |
| gemma-4-E4B-it | text generation | BF16 | 8.0B | 15.50 GiB | 29.14 GiB |
| Qwen3-30B-A3B-Thinking-2507MoE | text generation | Q8_0 | 30.5B | 34.05 GiB | 10.59 GiB |
| Qwen3-4B | text generation | BF16 | 4.0B | 12.81 GiB | 31.83 GiB |
| Qwen3-8B | text generation | BF16 | 8.2B | 20.60 GiB | 24.04 GiB |
| Laguna-XS-2.1MoE | text generation | Q8_0 | 33.4B | 35.32 GiB | 9.32 GiB |
| Llama-3.2-1B-Instruct | text generation | F16 | 1.2B | 4.11 GiB | 40.53 GiB |
| Qwen-AgentWorld-35B-A3BMoE | text generation | Q8_0 | 34.7B | 35.80 GiB | 8.84 GiB |
| gpt-oss-20bMoE | text generation | F16 | 21.5B | 14.40 GiB | 30.24 GiB |
| KAT-Coder-V2.5-DevMoE | text generation | Q8_0 | 34.7B | 35.81 GiB | 8.83 GiB |
| Qwen3-30B-A3BMoE | text generation | Q8_0 | 30.5B | 34.05 GiB | 10.59 GiB |
| llama-3-youko-8b | text generation | Q8_0 | 8.0B | 12.79 GiB | 31.85 GiB |
| Laguna-S-2.1MoE | text generation | UD-IQ3_XXS | 118B | 43.71 GiB | 0.93 GiB |
| Llama-3.1-8B-Instruct | text generation | F32 | 8.0B | 34.76 GiB | 9.88 GiB |
| ced-base | text generation | F32 | 86M | 1.16 GiB | 43.48 GiB |
| Qwen2.5-7B-Instruct | text generation | F16 | 7.6B | 16.80 GiB | 27.84 GiB |
| UI-TARS-1.5-7B | text generation | F16 | 8.3B | 16.80 GiB | 27.84 GiB |
| gemma-3-1b-it | text generation | F16 | 1000M | 2.81 GiB | 41.83 GiB |
| gemma-4-E2B-it-qat-q4_0-unquantized | text generation | BF16 | 5.1B | 9.83 GiB | 34.81 GiB |
| Qwen3-1.7B | text generation | BF16 | 2.0B | 8.08 GiB | 36.56 GiB |
| GLM-4.7-FlashMoE | text generation | Q8_0 | 31.2B | 32.12 GiB | 12.52 GiB |
| embeddinggemma-300m | text generation | F32 | 303M | 2.05 GiB | 42.59 GiB |
| Llama-3.2-3B-Instruct | text generation | F16 | 3.2B | 10.30 GiB | 34.34 GiB |
| Qwen3-0.6B | text generation | BF16 | 752M | 5.68 GiB | 38.96 GiB |
| Qwen3-14B | text generation | BF16 | 14.8B | 33.37 GiB | 11.27 GiB |
| Ornith-1.0-35BMoE | text generation | Q8_0 | 34.7B | 35.81 GiB | 8.83 GiB |
| Wan2.1-T2V-1.3B | text generation | Q8_0 | 1.4B | 12.18 GiB | 32.46 GiB |
| Qwen3-Coder-NextMoE | text generation | IQ4_XS | 79.7B | 43.70 GiB | 0.94 GiB |
| LFM2.5-1.2B-Instruct | text generation | BF16 | 1.2B | 3.37 GiB | 41.27 GiB |
| Qwen2.5-Coder-7B-Instruct | text generation | Q8_0 | 7.6B | 17.69 GiB | 26.95 GiB |
| Qwen2.5-32B-Instruct | text generation | Q8_0 | 32.8B | 41.33 GiB | 3.31 GiB |
| Qwen2.5-1.5B-Instruct | text generation | F16 | 1.5B | 4.56 GiB | 40.08 GiB |
| Jan-v3-4B-base-instruct | text generation | BF16 | 4.4B | 13.53 GiB | 31.11 GiB |
| gemma-3-4b-it | text generation | BF16 | 4.3B | 8.85 GiB | 35.79 GiB |
| MiniCPM5-1B-Claude-Opus-Fable5-Thinking | text generation | F16 | 1.1B | 3.55 GiB | 41.09 GiB |
| Ornith-1.0-9B | text generation | BF16 | 9.2B | 18.98 GiB | 25.66 GiB |
| Agents-A1MoE | text generation | Q8_0 | 35.1B | 35.80 GiB | 8.84 GiB |
| gemma-4-26B-A4B-it-ultra-uncensored-hereticMoE | text generation | Q8_0 | 25.8B | 27.35 GiB | 17.29 GiB |
| Qwen2.5-Coder-32B-Instruct | text generation | Q4_0 | 32.8B | 43.62 GiB | 1.02 GiB |
| Qwen2.5-Coder-14B-Instruct | text generation | Q8_0 | 14.8B | 36.09 GiB | 8.55 GiB |
| Qwen3-4B-Instruct-2507 | text generation | F16 | 4.0B | 12.81 GiB | 31.83 GiB |
| granite-4.1-3b | text generation | BF16 | 3.4B | 9.65 GiB | 34.99 GiB |
| Qwen2.5-3B-Instruct | text generation | F32 | 3.1B | 13.44 GiB | 31.20 GiB |
| Phi-3.5-mini-instruct | text generation | F32 | 3.8B | 27.04 GiB | 17.60 GiB |
| gemma-2-2b-it | text generation | F32 | 2.6B | 12.41 GiB | 32.23 GiB |
| Qwen3-30B-A3B-Instruct-2507MoE | text generation | Q8_0 | 30.5B | 34.05 GiB | 10.59 GiB |
| MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking | text generation | F16 | 1.1B | 3.55 GiB | 41.09 GiB |
| Qwen2.5-0.5B-Instruct | text generation | F16 | 494M | 2.08 GiB | 42.56 GiB |
| DeepSeek-R1-0528-Qwen3-8B | text generation | BF16 | 8.2B | 20.60 GiB | 24.04 GiB |
| Qwen3-32B | text generation | Q8_0 | 32.8B | 41.32 GiB | 3.32 GiB |
| Qwen3-VL-8B-Instruct | text generation | BF16 | 8.8B | 20.60 GiB | 24.04 GiB |
| Sugoi-14B-Ultra-HF | text generation | F16 | 14.8B | 34.36 GiB | 10.28 GiB |
| gemma-4-12b-heretic-abliterated | text generation | Q8_0 | 12.0B | 15.11 GiB | 29.53 GiB |
| TinyLlama-1.1B-Chat-v1.0 | text generation | F16 | 1.1B | 3.53 GiB | 41.11 GiB |
| Qwen2.5-14B-Instruct | text generation | F16 | 14.8B | 34.36 GiB | 10.28 GiB |
| Mistral-Nemo-Instruct-2407 | text generation | F16 | 12.2B | 28.67 GiB | 15.97 GiB |
| Phi-4-mini-instruct | text generation | BF16 | 3.8B | 11.96 GiB | 32.68 GiB |
This page models a generic 48GB accelerator, so it answers what fits rather than how fast it runs. For tokens per second you need a specific card — pick one from hardware, where bandwidth is known.