NVIDIA · workstation
RTX A400
RTX A400 has 4 GB of VRAM at 96 GB/s — about 3.72 GiB usable after driver and compositor overhead. 573 of 2118 indexed models fit at 32K context with q4_0 KV.
Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
4 GB
GDDR6
Bandwidth
96 GB/s
64-bit bus
Tensor FP16
11 TF
dense
TDP
50 W
$135 MSRP
text 449vision language 47audio tts 19audio asr 36video 2embedding 20
What fits at 32K context
largest quantization that fits, per model · 573 of 2118 indexed
| Model | Best quant | Params | Weights● | KV● | Total in memory◐ | Headroom◐ | tok/s◐ |
|---|---|---|---|---|---|---|---|
| LFM2.5-Audio-1.5B-JP | F16 | 1.5B | 2.67 GiB | 0.00 GiB | 3.72 GiB | 0.00 GiB | 20±22% |
| umt5-xxl | Q3_K_S | 5.7B | 2.66 GiB | 0.00 GiB | 3.71 GiB | 0.01 GiB | 21±22% |
| medgemma-4b-it | Q4_K_L | 4.3B | 2.47 GiB | 0.22 GiB | 3.71 GiB | 0.01 GiB | 20±22% |
| amoral-gemma3-4B-v1 | Q4_K_L | 4.3B | 2.47 GiB | 0.22 GiB | 3.71 GiB | 0.01 GiB | 20±22% |
| gemma-3-4b-it-abliterated | Q4_K_L | 4.3B | 2.47 GiB | 0.22 GiB | 3.71 GiB | 0.01 GiB | 20±22% |
| gemma-3-4b-it | Q4_K_L | 4.3B | 2.47 GiB | 0.22 GiB | 3.71 GiB | 0.01 GiB | 20±22% |
| FrickFritz-4B | IQ4_XS | 4.7B | 2.42 GiB | 0.28 GiB | 3.71 GiB | 0.01 GiB | 20±22% |
| Newton-bot-3-VLM-mini-4B | IQ4_XS | 4.7B | 2.42 GiB | 0.28 GiB | 3.71 GiB | 0.01 GiB | 20±22% |
| qwen3.5-4b-agentic-coder-v4 | IQ4_XS | 4.7B | 2.42 GiB | 0.28 GiB | 3.71 GiB | 0.01 GiB | 20±22% |
| Myth-4B | IQ4_XS | 4.3B | 2.42 GiB | 0.28 GiB | 3.71 GiB | 0.01 GiB | 20±22% |
| Qwen3.5-4B-Uncensored | IQ4_XS | 4.7B | 2.42 GiB | 0.28 GiB | 3.71 GiB | 0.01 GiB | 20±22% |
| JOSIE-2-4B-Preview | IQ4_XS | 4.7B | 2.42 GiB | 0.28 GiB | 3.71 GiB | 0.01 GiB | 20±22% |
| Surogate-3.5-4B | IQ4_XS | 5.3B | 2.42 GiB | 0.28 GiB | 3.71 GiB | 0.01 GiB | 20±22% |
| Qwopus3.5-4B-v3 | IQ4_XS | 4.7B | 2.42 GiB | 0.28 GiB | 3.71 GiB | 0.01 GiB | 20±22% |
| Llama-3.2-3B-Instruct-roleplay-tuned | IQ4_XS | 3.2B | 1.71 GiB | 0.98 GiB | 3.71 GiB | 0.01 GiB | 20±22% |
| Llama-3.2-3B-Instruct-heretic-ablitered-uncensored | IQ4_XS | 3.2B | 1.71 GiB | 0.98 GiB | 3.71 GiB | 0.01 GiB | 20±22% |
| Llama3.2-3B-creative-writer-v0.1 | IQ4_XS | 3.2B | 1.71 GiB | 0.98 GiB | 3.71 GiB | 0.01 GiB | 20±22% |
| Firefly-V3.2 | IQ4_XS | 3.2B | 1.71 GiB | 0.98 GiB | 3.71 GiB | 0.01 GiB | 20±22% |
| Firefly-V3 | IQ4_XS | 3.2B | 1.71 GiB | 0.98 GiB | 3.71 GiB | 0.01 GiB | 20±22% |
| SmolLM3-3B | Q5_K_M | 3.1B | 2.06 GiB | 0.63 GiB | 3.71 GiB | 0.01 GiB | 20±22% |
| Qwen2.5-Omni-7B | UD-IQ2_M | 10.7B | 2.66 GiB | 0.00 GiB | 3.70 GiB | 0.02 GiB | 21±22% |
| Qwen3-4B | UD-IQ2_M | 4.0B | 1.43 GiB | 1.27 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| Jan-nano | UD-IQ2_M | 4.0B | 1.43 GiB | 1.27 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| Phi-4-mini-instruct-abliterated | Q2_K | 3.8B | 1.57 GiB | 1.13 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| Phi-4-mini-reasoning | Q2_K | 3.8B | 1.57 GiB | 1.13 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| Phi-4-mini-instruct | Q2_K | 3.8B | 1.57 GiB | 1.13 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| orpheus-3b-0.1-pretrained | IQ3_XS | 3.8B | 1.71 GiB | 0.98 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| Qwen3-VL-4B-Thinking | UD-IQ2_M | 4.4B | 1.43 GiB | 1.27 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| Qwen3-VL-4B-Instruct | UD-IQ2_M | 4.4B | 1.43 GiB | 1.27 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| Qwen3-4B-Thinking-2507 | UD-IQ2_M | 4.0B | 1.43 GiB | 1.27 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| Jan-nano-128k | UD-IQ2_M | 4.0B | 1.43 GiB | 1.27 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| Qwen3-4B-Instruct-2507 | UD-IQ2_M | 4.0B | 1.43 GiB | 1.27 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| Yi-Coder-1.5B-Chat | Q5_K_M | 1.5B | 1.02 GiB | 1.69 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| Yi-Coder-1.5B | Q5_K_M | 1.5B | 1.02 GiB | 1.69 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| Dolphin3.0-Llama3.2-3B | IQ4_XS | 3.2B | 1.70 GiB | 0.98 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| Llama-Doctor-3.2-3B-Instruct | I1-IQ4_XS | 3.2B | 1.70 GiB | 0.98 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| Llama-Song-Stream-3B-Instruct | IQ4_XS | 3.2B | 1.70 GiB | 0.98 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| llama-3.2-Korean-Bllossom-3B | IQ4_XS | 3.2B | 1.70 GiB | 0.98 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| Llama-3.2-3B-Instruct | IQ4_XS | 3.2B | 1.70 GiB | 0.98 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| llama-3.2-3b-instruct | IQ4_XS | 3.2B | 1.70 GiB | 0.98 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| Hermes-3-Llama-3.2-3B | IQ4_XS | 3.2B | 1.70 GiB | 0.98 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| Darwin-4B-Chimera | I1-Q4_K_M | 4.0B | 2.33 GiB | 0.36 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| LFM2-2.6B | Q8_0 | 2.6B | 2.55 GiB | 0.14 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| LFM2-2.6B-Transcript | Q8_0 | 2.6B | 2.55 GiB | 0.14 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| LFM2-VL-3B | Q8_0 | 3.0B | 2.55 GiB | 0.14 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| Gemma-3-4b-it-Uncensored-DBL-X | I1-Q4_K_S | 4.7B | 2.41 GiB | 0.26 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| Dolphin3.0-Qwen2.5-3b | Q6_K | 3.1B | 2.36 GiB | 0.32 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| Qwen2.5-Coder-3B-Instruct-abliterated | I1-Q6_K | 3.1B | 2.36 GiB | 0.32 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| GRM-Kerlin-3b-Abliterated | I1-Q6_K | 3.1B | 2.36 GiB | 0.32 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| Mythos-nano | I1-Q6_K | 3.1B | 2.36 GiB | 0.32 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| MATE-3B | I1-Q6_K | 3.1B | 2.36 GiB | 0.32 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| Mythos-nano-OBLITERATED | I1-Q6_K | 3.1B | 2.36 GiB | 0.32 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| Qwen2.5-3B-Instruct-Uncensored | I1-Q6_K | 3.1B | 2.36 GiB | 0.32 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| Nanonets-OCR-s | Q6_K | 3.8B | 2.36 GiB | 0.32 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| Qwen2.5-Coder-3B | Q6_K | 3.1B | 2.36 GiB | 0.32 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| raspberry-3B | Q6_K | 3.1B | 2.36 GiB | 0.32 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| VibeThinker-3B-OBLITERATED | I1-Q6_K | 3.1B | 2.36 GiB | 0.32 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| VibeThinker-3B | Q6_K | 3.1B | 2.36 GiB | 0.32 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| Fourier-Qwen2.5-VL-3B-0.67 | I1-Q6_K | 3.8B | 2.36 GiB | 0.32 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| Qwen2.5-VL-3B-Instruct | Q6_K | 3.8B | 2.36 GiB | 0.32 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.
Questions people ask
- What AI models can a RTX A400 run?
- 573 of 2118 indexed open-weight models fit a RTX A400 at 32,768 context with q4_0 KV cache, the largest being LFM2.5-Audio-1.5B-JP at F16. That covers text, vision-language, image, video and speech models.
- How much usable memory does a RTX A400 actually have?
- Its nameplate is 4 GB, but about 3.72 GiB is available to a model once driver and compositor overhead is accounted for.
- Is a RTX A400 fast for local AI?
- Its memory bandwidth is 96 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.