NVIDIA · workstation

RTX A400

RTX A400 has 4 GB of VRAM at 96 GB/s — about 3.72 GiB usable after driver and compositor overhead. 834 of 2118 indexed models fit at 4K context with q4_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
4 GB
GDDR6
Bandwidth
96 GB/s
64-bit bus
Tensor FP16
11 TF
dense
TDP
50 W
$135 MSRP
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 692vision language 58audio tts 19embedding 25video 2image 1audio asr 37

What fits at 4K context

largest quantization that fits, per model · 834 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
LFM2.5-Audio-1.5B-JPF161.5B2.67 GiB0.00 GiB3.72 GiB0.00 GiB20±22%
Vikhr-Gemma-2B-instructQ8_02.6B2.59 GiB0.11 GiB3.72 GiB0.00 GiB20±22%
gemma-2-2b-it-abliteratedQ8_02.6B2.59 GiB0.11 GiB3.72 GiB0.00 GiB20±22%
gemma-2-2b-itQ8_02.6B2.59 GiB0.11 GiB3.72 GiB0.00 GiB20±22%
Gemmasutra-Mini-2B-v1Q8_02.6B2.59 GiB0.11 GiB3.72 GiB0.00 GiB20±22%
Holo-3.1-4BIQ4_XS5.2B2.67 GiB0.04 GiB3.72 GiB0.00 GiB20±22%
ToriiGate-0.5IQ4_XS5.2B2.67 GiB0.04 GiB3.72 GiB0.00 GiB20±22%
granite-4.0-h-1bBF161.5B2.73 GiB0.01 GiB3.72 GiB0.00 GiB20±22%
Nanbeige4.1-3BQ5_K_M3.9B2.63 GiB0.07 GiB3.72 GiB0.00 GiB20±22%
Yi-6B-ChatI1-IQ3_M6.1B2.62 GiB0.07 GiB3.72 GiB0.00 GiB20±22%
GrammarCoder-7B-BaseI1-IQ2_M7.6B2.60 GiB0.06 GiB3.72 GiB0.00 GiB21±22%
MARTHA-LXVII.8B_QWEN-3.5-3.6_prune_9b-3.6_baseI1-IQ2_XXS8.1B2.65 GiB0.03 GiB3.71 GiB0.01 GiB20±22%
umt5-xxlQ3_K_S5.7B2.66 GiB0.00 GiB3.71 GiB0.01 GiB21±22%
dolphin-2.9.3-mistral-7B-32kI1-Q2_K7.2B2.54 GiB0.14 GiB3.71 GiB0.01 GiB20±22%
Mistral-7B-v0.3Q2_K7.2B2.54 GiB0.14 GiB3.71 GiB0.01 GiB20±22%
Mistral-7B-Instruct-v0.3-ParasiteI1-Q2_K7.2B2.54 GiB0.14 GiB3.71 GiB0.01 GiB20±22%
Mistral-7B-Instruct-v0.3-JbliteratedI1-Q2_K7.2B2.54 GiB0.14 GiB3.71 GiB0.01 GiB20±22%
Mistral-7B-Instruct-v0.3Q2_K7.2B2.54 GiB0.14 GiB3.71 GiB0.01 GiB20±22%
Mistral-7B-v0.3-Chinese-ChatQ2_K7.2B2.54 GiB0.14 GiB3.71 GiB0.01 GiB20±22%
Mathstral-7B-v0.1Q2_K7.2B2.54 GiB0.14 GiB3.71 GiB0.01 GiB20±22%
Hunyuan-7B-InstructIQ2_M7.5B2.53 GiB0.14 GiB3.71 GiB0.01 GiB20±22%
granite-3.1-8b-instructIQ2_XS8.2B2.50 GiB0.18 GiB3.71 GiB0.01 GiB20±22%
zeta-2.1I1-IQ2_XS8.3B2.53 GiB0.14 GiB3.71 GiB0.01 GiB21±22%
orpheus-3b-0.1-pretrainedQ5_K_S3.8B2.58 GiB0.12 GiB3.71 GiB0.01 GiB20±22%
SciPhi-Self-RAG-Mistral-7B-32kKV unresolvedI1-Q2_K7.2B2.53 GiB0.14 GiB3.71 GiB0.01 GiB20±22%
dolphin-2.2.1-mistral-7bKV unresolvedI1-Q2_K7.2B2.53 GiB0.14 GiB3.71 GiB0.01 GiB20±22%
OpenChat-3.5-7B-Qwen-v2.0KV unresolvedI1-Q2_K7.2B2.53 GiB0.14 GiB3.71 GiB0.01 GiB20±22%
openchat-3.5-0106KV unresolvedI1-Q2_K7.2B2.53 GiB0.14 GiB3.71 GiB0.01 GiB20±22%
CapybaraHermes-2.5-Mistral-7BKV unresolvedQ2_K7.2B2.53 GiB0.14 GiB3.71 GiB0.01 GiB20±22%
dolphin-2.8-mistral-7b-v02Q2_K7.2B2.53 GiB0.14 GiB3.71 GiB0.01 GiB20±22%
Mistral-7B-v0.2Q2_K7.2B2.53 GiB0.14 GiB3.71 GiB0.01 GiB20±22%
Mistral-7B-Instruct-v0.1KV unresolvedI1-Q2_K7.2B2.53 GiB0.14 GiB3.71 GiB0.01 GiB20±22%
Mistral-7B-Instruct-v0.2I1-Q2_K7.2B2.53 GiB0.14 GiB3.71 GiB0.01 GiB20±22%
ContextualKunoichi_KTO-7BI1-Q2_K7.2B2.53 GiB0.14 GiB3.71 GiB0.01 GiB20±22%
xLAM-7b-rI1-Q2_K7.2B2.53 GiB0.14 GiB3.71 GiB0.01 GiB20±22%
mistral-7b-uncensoredKV unresolvedQ2_K7.2B2.53 GiB0.14 GiB3.71 GiB0.01 GiB20±22%
MegaBeam-Mistral-7B-512kQ2_K7.2B2.53 GiB0.14 GiB3.71 GiB0.01 GiB20±22%
Ninja-v1-RP-WIPKV unresolvedI1-Q2_K7.2B2.53 GiB0.14 GiB3.71 GiB0.01 GiB20±22%
BioMistral-7BKV unresolvedQ2_K7.2B2.53 GiB0.14 GiB3.71 GiB0.01 GiB20±22%
SpydazWeb_AI_CyberTron_Ultra_7bKV unresolvedQ2_K7.2B2.53 GiB0.14 GiB3.71 GiB0.01 GiB20±22%
Kunoichi-DPO-v2-7BKV unresolvedQ2_K7.2B2.53 GiB0.14 GiB3.71 GiB0.01 GiB20±22%
INTELLECT-1-InstructI1-IQ1_M10.2B2.48 GiB0.18 GiB3.71 GiB0.01 GiB21±22%
DeepHat-V1-7B-Heretic-AbliteratedI1-IQ2_M7.6B2.59 GiB0.06 GiB3.71 GiB0.01 GiB21±22%
ShizhenGPT-7B-VLI1-IQ2_M8.3B2.59 GiB0.06 GiB3.71 GiB0.01 GiB21±22%
DeepHat-V1-7BIQ2_M7.6B2.59 GiB0.06 GiB3.71 GiB0.01 GiB21±22%
HuatuoGPT-o1-7BI1-IQ2_M7.6B2.59 GiB0.06 GiB3.71 GiB0.01 GiB21±22%
MathSmith-DS-Qwen-7B-LongCoTI1-IQ2_M7.6B2.59 GiB0.06 GiB3.71 GiB0.01 GiB21±22%
AstraGPTCoder-7BI1-IQ2_M7.6B2.59 GiB0.06 GiB3.71 GiB0.01 GiB21±22%
Qwen2.5-Coder-7B-Instruct-Ghidra-v2I1-IQ2_M7.6B2.59 GiB0.06 GiB3.71 GiB0.01 GiB21±22%
EsDrac-v1-7BI1-IQ2_M7.6B2.59 GiB0.06 GiB3.71 GiB0.01 GiB21±22%
Hemlock-Apothecary-7B-GRPO-e3I1-IQ2_M7.6B2.59 GiB0.06 GiB3.71 GiB0.01 GiB21±22%
openhands-lm-7b-v0.1I1-IQ2_M7.6B2.59 GiB0.06 GiB3.71 GiB0.01 GiB21±22%
Hemlock2-Coder-7B-GRPOI1-IQ2_M7.6B2.59 GiB0.06 GiB3.71 GiB0.01 GiB21±22%
shellwhiz-7bI1-IQ2_M7.6B2.59 GiB0.06 GiB3.71 GiB0.01 GiB21±22%
Qwen2.5-Coder-7B-Instruct-abliteratedI1-IQ2_M7.6B2.59 GiB0.06 GiB3.71 GiB0.01 GiB21±22%
Qwen2.5-Coder-7B-Instruct-OBLITERATED-advancedI1-IQ2_M7.6B2.59 GiB0.06 GiB3.71 GiB0.01 GiB21±22%
Qwen-STEM-Specialist-7BI1-IQ2_M7.6B2.59 GiB0.06 GiB3.71 GiB0.01 GiB21±22%
VulnLLM-R-7BI1-IQ2_M7.6B2.59 GiB0.06 GiB3.71 GiB0.01 GiB21±22%
Garnet-OCR-7B-0422I1-IQ2_M8.3B2.59 GiB0.06 GiB3.71 GiB0.01 GiB21±22%
UwU-7B-InstructI1-IQ2_M7.6B2.59 GiB0.06 GiB3.71 GiB0.01 GiB21±22%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a RTX A400 run?
834 of 2118 indexed open-weight models fit a RTX A400 at 4,096 context with q4_0 KV cache, the largest being LFM2.5-Audio-1.5B-JP at F16. That covers text, vision-language, image, video and speech models.
How much usable memory does a RTX A400 actually have?
Its nameplate is 4 GB, but about 3.72 GiB is available to a model once driver and compositor overhead is accounted for.
Is a RTX A400 fast for local AI?
Its memory bandwidth is 96 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.