NVIDIA · workstation

RTX A400

RTX A400 has 4 GB of VRAM at 96 GB/s — about 3.72 GiB usable after driver and compositor overhead. 791 of 2118 indexed models fit at 4K context with q8_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
4 GB
GDDR6
Bandwidth
96 GB/s
64-bit bus
Tensor FP16
11 TF
dense
TDP
50 W
$135 MSRP
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 654embedding 25vision language 53audio tts 19audio asr 37image 1video 2

What fits at 4K context

largest quantization that fits, per model · 791 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
LFM2-2.6B-ExpQ8_02.6B2.68 GiB0.03 GiB3.72 GiB0.00 GiB20±22%
Qwen3-4B-BaseQ4_04.0B2.41 GiB0.30 GiB3.72 GiB0.00 GiB20±22%
LFM2.5-Audio-1.5B-JPF161.5B2.67 GiB0.00 GiB3.72 GiB0.00 GiB20±22%
OLMoE-1B-7B-0924-InstructMoEQ2_K_L6.9B2.48 GiB0.27 GiB3.72 GiB0.00 GiB44±37%
Nemotron-3-Embed-8B-BF16IQ2_XS8.0B2.39 GiB0.28 GiB3.72 GiB0.00 GiB20±22%
umt5-xxlQ3_K_S5.7B2.66 GiB0.00 GiB3.71 GiB0.01 GiB21±22%
Apertus-8B-Instruct-2509UD-IQ2_XXS8.1B2.38 GiB0.27 GiB3.71 GiB0.01 GiB21±22%
Phi-3.5-mini-instructQ3_K_L3.8B1.90 GiB0.80 GiB3.71 GiB0.01 GiB20±22%
salamandra-7b-instruct-2606I1-IQ2_XXS7.8B2.41 GiB0.27 GiB3.71 GiB0.01 GiB20±22%
Qwen2.5-Omni-7BUD-IQ2_M10.7B2.66 GiB0.00 GiB3.70 GiB0.02 GiB21±22%
Dolphin3.0-Llama3.2-3BQ6_K3.2B2.46 GiB0.23 GiB3.70 GiB0.02 GiB20±22%
Llama-Doctor-3.2-3B-InstructI1-Q6_K3.2B2.46 GiB0.23 GiB3.70 GiB0.02 GiB20±22%
Llama-Song-Stream-3B-InstructQ6_K3.2B2.46 GiB0.23 GiB3.70 GiB0.02 GiB20±22%
Llama-3.2-3B-Instruct-roleplay-tunedI1-Q6_K3.2B2.46 GiB0.23 GiB3.70 GiB0.02 GiB20±22%
Llama-3.2-3B-Instruct-heretic-ablitered-uncensoredI1-Q6_K3.2B2.46 GiB0.23 GiB3.70 GiB0.02 GiB20±22%
llama-3.2-Korean-Bllossom-3BQ6_K3.2B2.46 GiB0.23 GiB3.70 GiB0.02 GiB20±22%
llama-3.2-3b-instruct-bnb-4bitQ6_K3.3B2.46 GiB0.23 GiB3.70 GiB0.02 GiB20±22%
Llama-3.2-3B-InstructQ6_K3.2B2.46 GiB0.23 GiB3.70 GiB0.02 GiB20±22%
Llama-3.2-3BQ6_K3.2B2.46 GiB0.23 GiB3.70 GiB0.02 GiB20±22%
llama-3.2-3b-instructQ6_K3.2B2.46 GiB0.23 GiB3.70 GiB0.02 GiB20±22%
Llama3.2-3B-creative-writer-v0.1I1-Q6_K3.2B2.46 GiB0.23 GiB3.70 GiB0.02 GiB20±22%
Firefly-V3.2I1-Q6_K3.2B2.46 GiB0.23 GiB3.70 GiB0.02 GiB20±22%
Firefly-V3I1-Q6_K3.2B2.46 GiB0.23 GiB3.70 GiB0.02 GiB20±22%
Hermes-3-Llama-3.2-3BQ6_K3.2B2.46 GiB0.23 GiB3.70 GiB0.02 GiB20±22%
INTELLECT-1-InstructI1-IQ1_S10.2B2.31 GiB0.35 GiB3.70 GiB0.02 GiB21±22%
NVIDIA-Nemotron-3-Nano-4B-BF16IQ3_M4.0B2.32 GiB0.35 GiB3.70 GiB0.02 GiB20±22%
granite-4.0-7B-A1B-Creative-v0.1MoEI1-IQ3_S6.7B2.71 GiB0.02 GiB3.70 GiB0.02 GiB71±37%
Gemma-3-4b-it-Uncensored-DBL-XI1-Q4_K_M4.7B2.52 GiB0.15 GiB3.69 GiB0.03 GiB20±22%
Yi-6B-ChatI1-IQ3_S6.1B2.53 GiB0.13 GiB3.69 GiB0.03 GiB21±22%
moondream2F161.9B2.64 GiB0.00 GiB3.69 GiB0.03 GiB21±22%
Qwen2.5-3BQ6_K3.1B2.60 GiB0.07 GiB3.69 GiB0.03 GiB20±22%
GRM-Kerlin-3bI1-Q6_K3.4B2.60 GiB0.07 GiB3.69 GiB0.03 GiB20±22%
Qwen2.5-Coder-3B-InstructQ6_K3.1B2.60 GiB0.07 GiB3.69 GiB0.03 GiB20±22%
Qwen2.5-3B-InstructQ6_K3.1B2.60 GiB0.07 GiB3.69 GiB0.03 GiB20±22%
LCO-Embedding-Omni-3B-2605Q6_K4.7B2.60 GiB0.07 GiB3.69 GiB0.03 GiB20±22%
Garnet-OCR-3B-0422I1-Q6_K4.1B2.60 GiB0.07 GiB3.69 GiB0.03 GiB20±22%
MiMo-VL-7B-RLI1-IQ2_XS8.3B2.36 GiB0.30 GiB3.68 GiB0.04 GiB21±22%
Kuwutu-7B-CYOA-v2I1-IQ2_XS7.6B2.36 GiB0.30 GiB3.68 GiB0.04 GiB21±22%
Yi-1.5-6B-ChatQ3_K_S6.1B2.52 GiB0.13 GiB3.68 GiB0.04 GiB21±22%
VoxCPM2Q8_02.3B2.63 GiB0.00 GiB3.68 GiB0.04 GiB21±22%
Voxtral-Mini-3B-2507Q4_14.7B2.42 GiB0.25 GiB3.68 GiB0.04 GiB21±22%
Ministral-3-8B-Instruct-2512-BF16-abliteratedI1-IQ2_XXS8.9B2.35 GiB0.28 GiB3.67 GiB0.05 GiB21±22%
Amaretto-8BI1-IQ2_XXS8.9B2.35 GiB0.28 GiB3.67 GiB0.05 GiB21±22%
Teuken-7B-instruct-research-v0.4I1-IQ1_M7.5B2.57 GiB0.07 GiB3.67 GiB0.05 GiB21±22%
granite-3.3-2b-instructQ8_02.5B2.51 GiB0.17 GiB3.67 GiB0.05 GiB20±22%
granite-3.2-2b-instructQ8_02.5B2.51 GiB0.17 GiB3.67 GiB0.05 GiB20±22%
granite-vision-3.2-2bQ8_03.0B2.51 GiB0.17 GiB3.67 GiB0.05 GiB20±22%
EXAONE-Deep-7.8BIQ2_M7.8B2.63 GiB0.00 GiB3.67 GiB0.05 GiB21±22%
EXAONE-3.5-7.8B-InstructIQ2_M7.8B2.63 GiB0.00 GiB3.67 GiB0.05 GiB21±22%
FrickFritz-4BI1-Q4_K_M4.7B2.59 GiB0.07 GiB3.67 GiB0.05 GiB21±22%
qwen3.5-4b-agentic-coder-v4I1-Q4_K_M4.7B2.59 GiB0.07 GiB3.67 GiB0.05 GiB21±22%
Newton-bot-3-VLM-mini-4BQ4_K_M4.7B2.59 GiB0.07 GiB3.67 GiB0.05 GiB21±22%
Myth-4BI1-Q4_K_M4.3B2.59 GiB0.07 GiB3.67 GiB0.05 GiB21±22%
Qwen3.5-4B-UncensoredI1-Q4_K_M4.7B2.59 GiB0.07 GiB3.67 GiB0.05 GiB21±22%
JOSIE-2-4B-PreviewI1-Q4_K_M4.7B2.59 GiB0.07 GiB3.67 GiB0.05 GiB21±22%
Surogate-3.5-4BI1-Q4_K_M5.3B2.59 GiB0.07 GiB3.67 GiB0.05 GiB21±22%
Qwopus3.5-4B-Coder-Fable5-v1Q4_K_M4.7B2.59 GiB0.07 GiB3.67 GiB0.05 GiB21±22%
Qwopus3.5-4B-v3Q4_K_M4.7B2.59 GiB0.07 GiB3.67 GiB0.05 GiB21±22%
LFM2.5-Queen-Opus-4.7-8B-A1BMoEI1-IQ2_M8.5B2.65 GiB0.02 GiB3.67 GiB0.05 GiB61±37%
LFM2.5-8B-A1B-KO-SFTMoEI1-IQ2_M8.5B2.65 GiB0.02 GiB3.67 GiB0.05 GiB61±37%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a RTX A400 run?
791 of 2118 indexed open-weight models fit a RTX A400 at 4,096 context with q8_0 KV cache, the largest being LFM2-2.6B-Exp at Q8_0. That covers text, vision-language, image, video and speech models.
How much usable memory does a RTX A400 actually have?
Its nameplate is 4 GB, but about 3.72 GiB is available to a model once driver and compositor overhead is accounted for.
Is a RTX A400 fast for local AI?
Its memory bandwidth is 96 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.
RTX A400 — what AI models can it run locally? — ossmodeldb