NVIDIA · workstation

RTX A2000

RTX A2000 has 6 GB of VRAM at 288 GB/s — about 5.58 GiB usable after driver and compositor overhead. 581 of 2118 indexed models fit at 64K context with q8_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
6 GB
GDDR6
Bandwidth
288 GB/s
192-bit bus
Tensor FP16
32 TF
dense
TDP
70 W
$449 MSRP
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 448embedding 21vision language 57audio asr 36audio tts 17video 2

What fits at 64K context

largest quantization that fits, per model · 581 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
Llama-3.2-3B-InstructUD-IQ1_S3.2B0.85 GiB3.72 GiB5.58 GiB0.00 GiB36±22%
Nanbeige4.2-3BQ2_K4.2B1.64 GiB2.92 GiB5.58 GiB0.00 GiB36±22%
Qwen2.5-3BQ8_03.1B3.37 GiB1.20 GiB5.58 GiB0.00 GiB36±22%
GRM-Kerlin-3bQ8_03.4B3.37 GiB1.20 GiB5.58 GiB0.00 GiB36±22%
Qwen2.5-Coder-3B-InstructQ8_03.1B3.37 GiB1.20 GiB5.58 GiB0.00 GiB36±22%
Qwen2.5-3B-InstructQ8_03.1B3.37 GiB1.20 GiB5.58 GiB0.00 GiB36±22%
LCO-Embedding-Omni-3B-2605Q8_04.7B3.37 GiB1.20 GiB5.58 GiB0.00 GiB36±22%
OpenClaude-1.7B-MergedIQ3_M1.7B0.86 GiB3.72 GiB5.57 GiB0.01 GiB36±22%
Garnet-OCR-3B-0422Q8_04.1B3.37 GiB1.20 GiB5.57 GiB0.01 GiB36±22%
Qwen2.5-VL-7B-InstructUD-IQ2_M8.3B2.66 GiB1.86 GiB5.57 GiB0.01 GiB36±22%
glm4.1v-9b-base-sftI1-IQ2_XXS10.3B3.21 GiB1.33 GiB5.57 GiB0.01 GiB36±22%
MARTHA-LXVII.8B_QWEN-3.5-3.6_prune_9b-3.6_baseI1-IQ3_XS8.1B3.61 GiB0.93 GiB5.57 GiB0.01 GiB36±22%
LFM2-8B-A1BMoEIQ4_XS8.3B4.18 GiB0.40 GiB5.57 GiB0.01 GiB79±37%
Felldude-Uncensored-Ministral3-3B-bf16I1-IQ2_XS3.8B1.10 GiB3.45 GiB5.57 GiB0.01 GiB36±22%
Amaretto-3BI1-IQ2_XS4.3B1.10 GiB3.45 GiB5.57 GiB0.01 GiB36±22%
GrammarCoder-7B-BaseI1-Q2_K_S7.6B2.65 GiB1.86 GiB5.56 GiB0.02 GiB36±22%
Yi-6B-ChatI1-IQ3_XS6.1B2.41 GiB2.13 GiB5.56 GiB0.02 GiB36±22%
nomic-embed-codeQ2_K7.1B2.64 GiB1.86 GiB5.56 GiB0.02 GiB36±22%
DeepHat-V1-7B-Heretic-AbliteratedI1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
ShizhenGPT-7B-VLI1-Q2_K_S8.3B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
HuatuoGPT-o1-7BI1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
MathSmith-DS-Qwen-7B-LongCoTI1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
AstraGPTCoder-7BI1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
Qwen2.5-Coder-7B-Instruct-Ghidra-v2I1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
EsDrac-v1-7BI1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
Hemlock-Apothecary-7B-GRPO-e3I1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
openhands-lm-7b-v0.1I1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
Hemlock2-Coder-7B-GRPOI1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
shellwhiz-7bI1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
Qwen2.5-Coder-7B-Instruct-OBLITERATED-advancedI1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
Qwen-STEM-Specialist-7BI1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
VulnLLM-R-7BI1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
Garnet-OCR-7B-0422I1-Q2_K_S8.3B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
UwU-7B-InstructI1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
Video-R1-7BI1-Q2_K_S8.3B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
HARC-Qwen2.5-7B-InstructI1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
Qwen2.5-Coder-7B-AbliteratedI1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
Bozdogan-7BI1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
Crazy-AI-ModelI1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
turbo-ai-7bI1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
DeepSeek-R1-Distill-Qwen-7B-abliterated-v2I1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
Ghosty-7BI1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
SP-7BI1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
Qwen2.5-Coder-7B-Instruct-UncensoredI1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
Qwen2.5-VL-7B-Instruct-abliteratedI1-Q2_K_S8.3B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
DeepSeek-R1-Distill-Qwen-8B-AbliteratedI1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
Qwen2.5-7BQ2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
Qwen2.5-VL-7B-Instruct-hereticI1-Q2_K_S8.3B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
AWARES-Qwen2.5-VL-7BI1-Q2_K_S8.3B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
olmOCR-2-7B-1025I1-Q2_K_S8.3B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
Med-RwRI1-Q2_K_S8.3B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
SpatialThinker-7BI1-Q2_K_S8.3B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
MQ-Coldbrew-BaseI1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
Kepler-Reasoning-7BI1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
DeepSeek-R1-Distill-Qwen-7B-Uncensored-ReasonerI1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
DeepSeek-R1-Distill-Qwen-7B-UncensoredI1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
DeepSeek-R1-STEM-Coder-7BI1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
Kepler-8B-Instruct-v2I1-Q2_K_S7.6B2.64 GiB1.86 GiB5.55 GiB0.03 GiB36±22%
Teuken-7B-instruct-research-v0.4I1-IQ3_XS7.5B3.44 GiB1.06 GiB5.55 GiB0.03 GiB36±22%
Supertron2-Reranker-2BI1-IQ3_M2.1B0.83 GiB3.72 GiB5.55 GiB0.03 GiB36±22%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a RTX A2000 run?
581 of 2118 indexed open-weight models fit a RTX A2000 at 65,536 context with q8_0 KV cache, the largest being Llama-3.2-3B-Instruct at UD-IQ1_S. That covers text, vision-language, image, video and speech models.
How much usable memory does a RTX A2000 actually have?
Its nameplate is 6 GB, but about 5.58 GiB is available to a model once driver and compositor overhead is accounted for.
Is a RTX A2000 fast for local AI?
Its memory bandwidth is 288 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.