NVIDIA · consumer

GeForce RTX 2060

GeForce RTX 2060 has 6 GB of VRAM at 336 GB/s — about 5.58 GiB usable after driver and compositor overhead. 1228 of 2118 indexed models fit at 8K context with q4_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
6 GB
GDDR6
Bandwidth
336 GB/s
192-bit bus
Tensor FP16
52 TF
dense
TDP
160 W
$349 MSRP
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
embedding 26text 1048vision language 93audio asr 38audio tts 19image 1video 3

What fits at 8K context

largest quantization that fits, per model · 1228 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
nomic-embed-codeQ5_K_S7.1B4.60 GiB0.12 GiB5.58 GiB0.00 GiB50±12.9%
gemma-4-E4B-uncensoredIQ4_XS7.9B4.71 GiB0.05 GiB5.58 GiB0.00 GiB50±12.9%
gemma-4-E4B-it-qat-heretic_decensoredIQ4_XS7.9B4.71 GiB0.05 GiB5.58 GiB0.00 GiB50±12.9%
gemma-4-E4B-it-QAT-SOMPOA-heresyIQ4_XS7.9B4.71 GiB0.05 GiB5.58 GiB0.00 GiB50±12.9%
gemma-3-12b-it-vl-Gemini-3-Pro-Preview-Heretic-Uncensored-ThinkingI1-IQ3_XXS12.2B4.46 GiB0.27 GiB5.57 GiB0.01 GiB50±12.9%
gemma-3-12b-it-vl-Deepseek-v3.1-Heretic-Uncensored-ThinkingI1-IQ3_XXS12.2B4.46 GiB0.27 GiB5.57 GiB0.01 GiB50±12.9%
gemma-3-12b-it-vl-GLM-4.7-Flash-Heretic-Uncensored-ThinkingI1-IQ3_XXS12.2B4.46 GiB0.27 GiB5.57 GiB0.01 GiB50±12.9%
gemma-3-12b-it-ultra-uncensored-hereticIQ3_XXS12.2B4.46 GiB0.27 GiB5.57 GiB0.01 GiB50±12.9%
Floppa-12B-Gemma3-UncensoredI1-IQ3_XXS12.2B4.46 GiB0.27 GiB5.57 GiB0.01 GiB50±12.9%
gemma-3-12b-it-hereticI1-IQ3_XXS12.2B4.46 GiB0.27 GiB5.57 GiB0.01 GiB50±12.9%
gemma-3-12b-it-abliteratedIQ3_XXS12.2B4.46 GiB0.27 GiB5.57 GiB0.01 GiB50±12.9%
canary-qwen-2.5bBF162.6B4.73 GiB0.00 GiB5.57 GiB0.01 GiB50±12.9%
EXAONE-Deep-7.8BQ4_K_L7.8B4.73 GiB0.00 GiB5.57 GiB0.01 GiB50±12.9%
EXAONE-3.5-7.8B-InstructQ4_K_L7.8B4.73 GiB0.00 GiB5.57 GiB0.01 GiB50±12.9%
ERNIE-21B-A3B-Thinking-Gemini-3-Pro-High-Reasoning-V2I1-IQ1_M21.8B4.63 GiB0.12 GiB5.57 GiB0.01 GiB50±12.9%
ERNIE-21B-A3B-Claude-4.5-High-OPUS-ThinkingI1-IQ1_M21.8B4.63 GiB0.12 GiB5.57 GiB0.01 GiB50±12.9%
ERNIE-4.5-21B-A3B-ThinkingI1-IQ1_M21.8B4.63 GiB0.12 GiB5.57 GiB0.01 GiB50±12.9%
Qwen3-TTS-12Hz-0.6B-BaseQ4_K_M915M4.72 GiB0.00 GiB5.57 GiB0.01 GiB50±12.9%
VoxCPM2F162.3B4.72 GiB0.00 GiB5.57 GiB0.01 GiB50±12.9%
AfriqueGemma-12BI1-Q2_K_S12.2B4.45 GiB0.27 GiB5.57 GiB0.01 GiB50±12.9%
gemma-7bI1-IQ3_S8.5B3.71 GiB0.98 GiB5.57 GiB0.01 GiB51±12.9%
Hubble-4B-v1Q8_04.5B4.47 GiB0.28 GiB5.57 GiB0.01 GiB50±12.9%
Aura-4BQ8_04.5B4.47 GiB0.28 GiB5.57 GiB0.01 GiB50±12.9%
magnum-v2-4bQ8_04.5B4.47 GiB0.28 GiB5.57 GiB0.01 GiB50±12.9%
Impish_LLAMA_4BQ8_04.5B4.47 GiB0.28 GiB5.57 GiB0.01 GiB50±12.9%
Llama-3.1-Minitron-4B-Width-BaseQ8_04.5B4.47 GiB0.28 GiB5.57 GiB0.01 GiB50±12.9%
t5-v1_1-xxlQ2_K4.8B4.72 GiB0.00 GiB5.56 GiB0.02 GiB50±12.9%
granite-4.1-8bQ3_K_L8.8B4.38 GiB0.35 GiB5.56 GiB0.02 GiB50±12.9%
Anubis-Mini-8B-v1Q4_K_S8.0B4.44 GiB0.28 GiB5.56 GiB0.02 GiB50±12.9%
Nexa-AI-4x4B-InstructMoEI1-IQ3_XXS12.1B4.43 GiB0.32 GiB5.56 GiB0.02 GiB51±37%
gemma-3n-E4B-itQ5_K_M7.8B4.68 GiB0.05 GiB5.56 GiB0.02 GiB50±12.9%
gemma-4-E4B-it-hereticQ4_18.0B4.69 GiB0.05 GiB5.56 GiB0.02 GiB50±12.9%
Phi-3-mini-4k-instructKV unresolvedIQ2_XS3.8B3.91 GiB0.84 GiB5.56 GiB0.02 GiB50±12.9%
gemma-3-12b-it-abliterated-v2Q2_K11.8B4.44 GiB0.27 GiB5.56 GiB0.02 GiB50±12.9%
gemma-3-12b-itQ2_K12.2B4.44 GiB0.27 GiB5.56 GiB0.02 GiB50±12.9%
GLM-4.6V-FlashQ3_K_M10.3B4.63 GiB0.09 GiB5.56 GiB0.02 GiB50±12.9%
GLM-Z1-9B-0414Q3_K_M9.4B4.63 GiB0.09 GiB5.56 GiB0.02 GiB50±12.9%
glm4.1v-9b-base-sftI1-Q3_K_M10.3B4.63 GiB0.09 GiB5.56 GiB0.02 GiB50±12.9%
GLM-4-9B-0414Q3_K_M9.4B4.63 GiB0.09 GiB5.56 GiB0.02 GiB50±12.9%
GLM-4.1V-9B-ThinkingQ3_K_M10.3B4.63 GiB0.09 GiB5.56 GiB0.02 GiB50±12.9%
gemma-4-E4B-it-qat-q4_0-unquantized-hereticI1-IQ4_XS7.9B4.69 GiB0.05 GiB5.56 GiB0.02 GiB50±12.9%
gemma4-e4b-mahou-nsfwI1-IQ4_XS7.9B4.69 GiB0.05 GiB5.56 GiB0.02 GiB50±12.9%
gemma-4-E4B-it-mentalchat16kI1-IQ4_XS7.9B4.69 GiB0.05 GiB5.56 GiB0.02 GiB50±12.9%
gemma4-E4B-it-abliteratedI1-IQ4_XS7.9B4.69 GiB0.05 GiB5.56 GiB0.02 GiB50±12.9%
gemma-4-E4B-it-OBLITERATEDI1-IQ4_XS8.0B4.69 GiB0.05 GiB5.56 GiB0.02 GiB50±12.9%
dolphincoder-starcoder2-15bKV unresolvedI1-IQ2_XS16.0B4.49 GiB0.18 GiB5.56 GiB0.02 GiB51±12.9%
starcoder2-15bKV unresolvedIQ2_XS16.0B4.49 GiB0.18 GiB5.56 GiB0.02 GiB51±12.9%
LFM2-8B-A1BMoEQ4_K_L8.3B4.73 GiB0.03 GiB5.56 GiB0.02 GiB145±37%
Gemma-4-E4B-LuchadorQ3_K_L8.0B4.69 GiB0.05 GiB5.55 GiB0.03 GiB50±12.9%
Trinity-Nano-PreviewMoEQ6_K6.1B4.72 GiB0.06 GiB5.55 GiB0.03 GiB194±37%
gemma-2bF162.5B4.67 GiB0.04 GiB5.55 GiB0.03 GiB50±12.9%
legitus-instruct-v1I1-Q4_K_S8.1B4.40 GiB0.28 GiB5.55 GiB0.03 GiB51±12.9%
Apertus-8B-Instruct-2509I1-Q4_K_S8.1B4.40 GiB0.28 GiB5.55 GiB0.03 GiB51±12.9%
granite-3.3-8b-instructQ4_K_S8.2B4.36 GiB0.35 GiB5.55 GiB0.03 GiB50±12.9%
granite-3.2-8b-instructQ4_K_S8.2B4.36 GiB0.35 GiB5.55 GiB0.03 GiB50±12.9%
Ministral-3-8B-Instruct-2512-BF16-abliteratedIQ4_XS8.9B4.41 GiB0.30 GiB5.55 GiB0.03 GiB50±12.9%
Amaretto-8BIQ4_XS8.9B4.41 GiB0.30 GiB5.55 GiB0.03 GiB50±12.9%
Hunyuan-7B-InstructQ4_K_L7.5B4.42 GiB0.28 GiB5.54 GiB0.04 GiB51±12.9%
Qwen3-16B-A3BMoEUD-IQ1_S16.0B4.54 GiB0.21 GiB5.54 GiB0.04 GiB126±37%
deepseek-coder-6.7b-instructQ4_K_S6.7B3.59 GiB1.13 GiB5.54 GiB0.04 GiB50±12.9%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a GeForce RTX 2060 run?
1228 of 2118 indexed open-weight models fit a GeForce RTX 2060 at 8,192 context with q4_0 KV cache, the largest being nomic-embed-code at Q5_K_S. That covers text, vision-language, image, video and speech models.
How much usable memory does a GeForce RTX 2060 actually have?
Its nameplate is 6 GB, but about 5.58 GiB is available to a model once driver and compositor overhead is accounted for.
Is a GeForce RTX 2060 fast for local AI?
Its memory bandwidth is 336 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.