NVIDIA · workstation
RTX A2000
RTX A2000 has 6 GB of VRAM at 288 GB/s — about 5.58 GiB usable after driver and compositor overhead. 1211 of 2118 indexed models fit at 4K context with q8_0 KV.
Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
6 GB
GDDR6
Bandwidth
288 GB/s
192-bit bus
Tensor FP16
32 TF
dense
TDP
70 W
$449 MSRP
text 1035image 1vision language 90embedding 26audio asr 38audio tts 19video 2
What fits at 4K context
largest quantization that fits, per model · 1211 of 2118 indexed
| Model | Best quant | Params | Weights● | KV● | Total in memory◐ | Headroom◐ | tok/s◐ |
|---|---|---|---|---|---|---|---|
| deepseek-math-7b-instruct | IQ4_XS | 6.9B | 3.56 GiB | 1.00 GiB | 5.58 GiB | 0.00 GiB | 36±22% |
| Qwen3-Reranker-8B | I1-IQ4_XS | 8.2B | 4.25 GiB | 0.30 GiB | 5.58 GiB | 0.00 GiB | 36±22% |
| Janus-Pro-7B | IQ4_XS | 7.4B | 3.56 GiB | 1.00 GiB | 5.58 GiB | 0.00 GiB | 36±22% |
| deepseek-coder-7b-instruct-v1.5 | IQ4_XS | 6.9B | 3.56 GiB | 1.00 GiB | 5.58 GiB | 0.00 GiB | 36±22% |
| Qwythos-9B-v2 | Q3_K_S | 9.7B | 4.48 GiB | 0.07 GiB | 5.58 GiB | 0.00 GiB | 36±22% |
| Tess-4-9B | Q3_K_S | 9.7B | 4.48 GiB | 0.07 GiB | 5.58 GiB | 0.00 GiB | 36±22% |
| Parable-Granite-4.1-8B-Claude-Fable-5 | I1-Q3_K_L | 8.4B | 4.21 GiB | 0.33 GiB | 5.58 GiB | 0.00 GiB | 36±22% |
| SOLAR-10.7B-Instruct-v1.0 | I1-IQ3_XS | 10.7B | 4.14 GiB | 0.40 GiB | 5.58 GiB | 0.00 GiB | 36±22% |
| LFM2-8B-A1BMoE | Q4_K_S | 8.3B | 4.56 GiB | 0.02 GiB | 5.58 GiB | 0.00 GiB | 106±37% |
| Tiger-Gemma-12B-v3 | IQ2_S | 12.8B | 4.15 GiB | 0.38 GiB | 5.58 GiB | 0.00 GiB | 36±22% |
| AfriqueGemma-12B | I1-IQ2_S | 12.2B | 4.15 GiB | 0.38 GiB | 5.58 GiB | 0.00 GiB | 36±22% |
| INTELLECT-1-Instruct | Q2_K_L | 10.2B | 4.19 GiB | 0.35 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| gemma-4-E4B-uncensored | I1-Q3_K_M | 7.9B | 4.49 GiB | 0.07 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| gemma-4-E4B-it-qat-q4_0-unquantized-heretic | I1-Q3_K_M | 7.9B | 4.49 GiB | 0.07 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| gemma-4-E4B-it-qat-heretic_decensored | I1-Q3_K_M | 7.9B | 4.49 GiB | 0.07 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| gemma-4-E4B-it-QAT-SOMPOA-heresy | I1-Q3_K_M | 7.9B | 4.49 GiB | 0.07 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| gemma4-e4b-mahou-nsfw | I1-Q3_K_M | 7.9B | 4.49 GiB | 0.07 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| gemma-4-E4B-it-mentalchat16k | I1-Q3_K_M | 7.9B | 4.49 GiB | 0.07 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| gemma4-E4B-it-abliterated | I1-Q3_K_M | 7.9B | 4.49 GiB | 0.07 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| gemma-4-E4B-it-OBLITERATED | I1-Q3_K_M | 8.0B | 4.49 GiB | 0.07 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| Ministral-3-8B-Instruct-2512-BF16-abliterated | I1-Q3_K_L | 8.9B | 4.25 GiB | 0.28 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| Amaretto-8B | I1-Q3_K_L | 8.9B | 4.25 GiB | 0.28 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| gemma-3-12b-it-vl-Gemini-3-Pro-Preview-Heretic-Uncensored-Thinking | I1-Q2_K_S | 12.2B | 4.14 GiB | 0.38 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| gemma-3-12b-it-vl-Deepseek-v3.1-Heretic-Uncensored-Thinking | I1-Q2_K_S | 12.2B | 4.14 GiB | 0.38 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| gemma-3-12b-it-vl-GLM-4.7-Flash-Heretic-Uncensored-Thinking | I1-Q2_K_S | 12.2B | 4.14 GiB | 0.38 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| Floppa-12B-Gemma3-Uncensored | I1-Q2_K_S | 12.2B | 4.14 GiB | 0.38 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| gemma-3-12b-it-heretic | I1-Q2_K_S | 12.2B | 4.14 GiB | 0.38 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| codegeex4-all-9b | IQ2_XXS | 9.4B | 3.19 GiB | 1.33 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| gemma-4-E4B-it-heretic | Q4_K_S | 8.0B | 4.48 GiB | 0.07 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| Forsaken-Void-12B | I1-Q2_K_S | 12.2B | 4.19 GiB | 0.33 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Silver-Siren-ST-12B | I1-Q2_K_S | 12.2B | 4.19 GiB | 0.33 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Dans-PersonalityEngine-V1.3.0-12b | I1-Q2_K_S | 12.2B | 4.19 GiB | 0.33 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| MN-12B-Runeweaver-RP-RU | I1-Q2_K_S | 12.2B | 4.19 GiB | 0.33 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Impish_Bloodmoon_12B | I1-Q2_K_S | 12.2B | 4.19 GiB | 0.33 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Wayfarer-2-12B | I1-Q2_K_S | 12.2B | 4.19 GiB | 0.33 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Wayfarer-12B | I1-Q2_K_S | 12.2B | 4.19 GiB | 0.33 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Muse-12B | I1-Q2_K_S | 12.2B | 4.19 GiB | 0.33 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Rocinante-X-12B-v1-Heretic-Uncensored | I1-Q2_K_S | 12.2B | 4.19 GiB | 0.33 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Mistral-Heretica-12B | I1-Q2_K_S | 12.2B | 4.19 GiB | 0.33 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| arcee-fusion-lumaid-12B | I1-Q2_K_S | 12.2B | 4.19 GiB | 0.33 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Mistral-NeMo-12B-Abliterated | I1-Q2_K_S | 12.2B | 4.19 GiB | 0.33 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Captain-Eris_Violet-V0.420-12B | I1-Q2_K_S | 12.2B | 4.19 GiB | 0.33 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Rocinante-X-12B-v1-absolute-heresy | I1-Q2_K_S | 12.2B | 4.19 GiB | 0.33 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Rocinante-X-12B-v1 | I1-Q2_K_S | 12.2B | 4.19 GiB | 0.33 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Mistral-Nemo-2407-12B-Thinking-Claude-Gemini-GPT5.2-Uncensored-HERETIC | I1-Q2_K_S | 12.2B | 4.19 GiB | 0.33 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Dans-SakuraKaze-V1.0.0-12b | I1-Q2_K_S | 12.2B | 4.19 GiB | 0.33 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Mistral-Nemo-Inst-2407-12B-Thinking-Uncensored-HERETIC-HI-Claude-Opus | I1-Q2_K_S | 12.2B | 4.19 GiB | 0.33 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Mistral-Nemo-Instruct-2407-12B-Thinking-M-Claude-Opus-High-Reasoning | I1-Q2_K_S | 12.2B | 4.19 GiB | 0.33 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Mordant-12B-Think | I1-Q2_K_S | 12.2B | 4.19 GiB | 0.33 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Riverfish-Rocinante-12B-SFT-DPO | I1-Q2_K_S | 12.2B | 4.19 GiB | 0.33 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| MN-Violet-Lotus-12B-Heretic | I1-Q2_K_S | 12.2B | 4.19 GiB | 0.33 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Himeyuri-Magnum-12B-HereticMerge | I1-Q2_K_S | 12.2B | 4.19 GiB | 0.33 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Kinggaroo-12b-v1 | I1-Q2_K_S | 12.2B | 4.19 GiB | 0.33 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Nera_Noctis-12B | I1-Q2_K_S | 12.2B | 4.19 GiB | 0.33 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Peaceful-Days-12B | I1-Q2_K_S | 12.2B | 4.19 GiB | 0.33 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Blissful-Days-12B | I1-Q2_K_S | 12.2B | 4.19 GiB | 0.33 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| glm-4-9b-chat | IQ2_XXS | 9.4B | 3.19 GiB | 1.33 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| SambaLingo-Japanese-Chat | I1-Q3_K_L | 6.9B | 3.47 GiB | 1.06 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Tini-Cybersec-8B-A1BMoE | Q4_0 | 8.5B | 4.54 GiB | 0.02 GiB | 5.56 GiB | 0.02 GiB | 107±37% |
| Qwen3-14B | UD-IQ2_XXS | 14.8B | 4.16 GiB | 0.33 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.
Questions people ask
- What AI models can a RTX A2000 run?
- 1211 of 2118 indexed open-weight models fit a RTX A2000 at 4,096 context with q8_0 KV cache, the largest being deepseek-math-7b-instruct at IQ4_XS. That covers text, vision-language, image, video and speech models.
- How much usable memory does a RTX A2000 actually have?
- Its nameplate is 6 GB, but about 5.58 GiB is available to a model once driver and compositor overhead is accounted for.
- Is a RTX A2000 fast for local AI?
- Its memory bandwidth is 288 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.