NVIDIA · workstation

RTX A400

RTX A400 has 4 GB of VRAM at 96 GB/s — about 3.72 GiB usable after driver and compositor overhead. 736 of 2118 indexed models fit at 8K context with q8_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
4 GB
GDDR6
Bandwidth
96 GB/s
64-bit bus
Tensor FP16
11 TF
dense
TDP
50 W
$135 MSRP
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 604audio asr 36audio tts 19vision language 50embedding 25video 2

What fits at 8K context

largest quantization that fits, per model · 736 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
Phi-4-mini-reasoningQ4_K_S3.8B2.18 GiB0.53 GiB3.72 GiB0.00 GiB20±22%
Phi-4-mini-instructQ4_K_S3.8B2.18 GiB0.53 GiB3.72 GiB0.00 GiB20±22%
Voxtral-Mini-3B-2507Q4_04.7B2.21 GiB0.50 GiB3.72 GiB0.00 GiB20±22%
LFM2.5-Audio-1.5B-JPF161.5B2.67 GiB0.00 GiB3.72 GiB0.00 GiB20±22%
Ministral-3-8B-Reasoning-2512UD-IQ1_S8.9B2.11 GiB0.56 GiB3.72 GiB0.00 GiB20±22%
phi-2Q3_K_M2.8B1.38 GiB1.33 GiB3.72 GiB0.00 GiB20±22%
Darwin-4B-ChimeraI1-Q4_K_M4.0B2.33 GiB0.38 GiB3.72 GiB0.00 GiB20±22%
Nanbeige4.2-3BQ4_K_S4.2B2.33 GiB0.37 GiB3.72 GiB0.00 GiB20±22%
granite-8b-code-instruct-4kI1-IQ2_XXS8.1B2.08 GiB0.60 GiB3.72 GiB0.00 GiB20±22%
granite-8b-code-base-4kI1-IQ2_XXS8.1B2.08 GiB0.60 GiB3.72 GiB0.00 GiB20±22%
EVA-Yi-1.5-9B-32K-V1I1-IQ2_XXS8.8B2.29 GiB0.40 GiB3.71 GiB0.01 GiB20±22%
GrammarCoder-7B-BaseI1-IQ2_S7.6B2.43 GiB0.23 GiB3.71 GiB0.01 GiB21±22%
umt5-xxlQ3_K_S5.7B2.66 GiB0.00 GiB3.71 GiB0.01 GiB21±22%
granite-4.0-7B-A1B-Creative-v0.1MoEI1-IQ3_S6.7B2.71 GiB0.03 GiB3.71 GiB0.01 GiB68±37%
Llama-3.1-8B-InstructUD-IQ1_M8.0B2.13 GiB0.53 GiB3.71 GiB0.01 GiB21±22%
Llama-3.1-Nemotron-Nano-8B-v1UD-IQ1_M8.0B2.13 GiB0.53 GiB3.71 GiB0.01 GiB21±22%
DeepSeek-R1-Distill-Llama-8BUD-IQ1_M8.0B2.13 GiB0.53 GiB3.71 GiB0.01 GiB21±22%
orpheus-3b-0.1-ftQ5_K_M3.8B2.23 GiB0.46 GiB3.71 GiB0.01 GiB20±22%
Qwen2.5-Omni-7BUD-IQ2_M10.7B2.66 GiB0.00 GiB3.70 GiB0.02 GiB21±22%
DeepHat-V1-7B-Heretic-AbliteratedI1-IQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
ShizhenGPT-7B-VLI1-IQ2_S8.3B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
HuatuoGPT-o1-7BI1-IQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
MathSmith-DS-Qwen-7B-LongCoTI1-IQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
AstraGPTCoder-7BI1-IQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
Qwen2.5-Coder-7B-Instruct-Ghidra-v2I1-IQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
EsDrac-v1-7BI1-IQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
Hemlock-Apothecary-7B-GRPO-e3I1-IQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
openhands-lm-7b-v0.1I1-IQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
Hemlock2-Coder-7B-GRPOI1-IQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
shellwhiz-7bI1-IQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
Qwen2.5-Coder-7B-Instruct-abliteratedI1-IQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
Qwen2.5-Coder-7B-Instruct-OBLITERATED-advancedI1-IQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
Qwen-STEM-Specialist-7BI1-IQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
VulnLLM-R-7BI1-IQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
Garnet-OCR-7B-0422I1-IQ2_S8.3B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
UwU-7B-InstructI1-IQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
Video-R1-7BI1-IQ2_S8.3B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
HARC-Qwen2.5-7B-InstructI1-IQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
Qwen2.5-Coder-7B-AbliteratedI1-IQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
Bozdogan-7BI1-IQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
Crazy-AI-ModelI1-IQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
turbo-ai-7bI1-IQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
DeepSeek-R1-Distill-Qwen-7B-abliterated-v2I1-IQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
Ghosty-7BI1-IQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
SP-7BI1-IQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
Qwen2.5-Coder-7B-Instruct-UncensoredI1-IQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
Qwen2.5-VL-7B-Instruct-abliteratedI1-IQ2_S8.3B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
DeepSeek-R1-Distill-Qwen-8B-AbliteratedI1-IQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
Qwen2.5-7BIQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
Qwen2.5-VL-7B-Instruct-hereticI1-IQ2_S8.3B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
AWARES-Qwen2.5-VL-7BI1-IQ2_S8.3B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
olmOCR-2-7B-1025I1-IQ2_S8.3B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
Qwen2.5-7B-Instruct-UncensoredI1-IQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
Med-RwRI1-IQ2_S8.3B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
EVA-Qwen2.5-7B-v0.1I1-IQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
SpatialThinker-7BI1-IQ2_S8.3B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
Human-Like-Qwen2.5-7B-InstructI1-IQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
Qwen2-7B-InstructIQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
Hercules-5.0-Qwen2-7BIQ2_S7.6B2.42 GiB0.23 GiB3.70 GiB0.02 GiB21±22%
Qianfan-OCRQ3_K_M4.7B2.09 GiB0.60 GiB3.70 GiB0.02 GiB20±22%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a RTX A400 run?
736 of 2118 indexed open-weight models fit a RTX A400 at 8,192 context with q8_0 KV cache, the largest being Phi-4-mini-reasoning at Q4_K_S. That covers text, vision-language, image, video and speech models.
How much usable memory does a RTX A400 actually have?
Its nameplate is 4 GB, but about 3.72 GiB is available to a model once driver and compositor overhead is accounted for.
Is a RTX A400 fast for local AI?
Its memory bandwidth is 96 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.