NVIDIA · consumer

GeForce RTX 2060

GeForce RTX 2060 has 6 GB of VRAM at 336 GB/s — about 5.58 GiB usable after driver and compositor overhead. 1188 of 2118 indexed models fit at 16K context with q4_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
6 GB
GDDR6
Bandwidth
336 GB/s
192-bit bus
Tensor FP16
52 TF
dense
TDP
160 W
$349 MSRP
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
vision language 91text 1010audio asr 38audio tts 19embedding 26image 1video 3

What fits at 16K context

largest quantization that fits, per model · 1188 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
gemma-4-12B-itUD-IQ3_XXS12.0B4.32 GiB0.41 GiB5.58 GiB0.00 GiB50±12.9%
gemma-4-12B-it-hereticIQ3_XXS12.0B4.32 GiB0.41 GiB5.58 GiB0.00 GiB50±12.9%
Foundation-Sec-8B-Instruct-hereticIQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
MiniCPM-Llama3-V-2_5IQ4_XS8.5B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
llama3.1-heretic-unsensoredIQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
dolphin-2.9.4-llama3.1-8bIQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
Dolphin3.0-Llama3.1-8B-abliteratedIQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
Deepseek-R1-Distill-NSFW-RPv1IQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
Llama3-ChatQA-1.5-8BIQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
L3.1-Dark-Reasoning-LewdPlay-evo-Hermes-R1-Uncensored-8B-hereticIQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
Llama3.3-8B-Instruct-Thinking-Heretic-Uncensored-Claude-4.5-Opus-High-ReasoningIQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
Llama3.3-8B-Instruct-Thinking-Claude-4.5-Opus-High-ReasoningIQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
grok-oss-Apollyon-8B-hereticIQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
Llama3.1-GptDeluxe-8BIQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
SelfCite-8B-CC-SFTIQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
Llama-3.3-8B-Instruct-128K-JbliteratedIQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
ai-girlfriend-v2IQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
Llama-3.3-8B-Instruct-128K_AbliteratedIQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
grok-oss-Apollyon-8BIQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
Lumen-1.2.5IQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
HARC-Llama-3.1-8B-InstructIQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
grok-oss-Thanatos-8BIQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
L3-8B-Sunfall-v0.5-Stheno-v3.2IQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
OpenElla-NovelWriter-Requiem-AscendedIQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
llama-joycaption-beta-one-hf-llavaIQ4_XS8.5B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
calme-2.3-legalkit-8bIQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
DarkIdol-Llama-3.1-8B-Instruct-1.3-UncensoredIQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
Llama3.1-DilemmaDeluxe-8BIQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
DarkIdol-Llama-3.1-8B-Instruct-1.2-UncensoredIQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
Humanish-Roleplay-Llama-3.1-8BIQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
Llama-3.1-8B-kali-pentesterIQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
NeuralDaredevil-8B-abliteratedIQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
Llama-3.3-8B-InstructIQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
Hathor_Sofit-L3-8B-v1IQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
llama3-8B-DarkIdol-2.3-Uncensored-32KIQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
Hathor_Unstable-L3-8B-v0.3IQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
L3-8B-Stheno-v3.2IQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
Llama-SEA-LION-v2-8BIQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
Turkish-Llama-8b-Instruct-v0.1IQ4_XS8.0B4.18 GiB0.56 GiB5.58 GiB0.00 GiB50±12.9%
Apriel-1.6-15b-ThinkerI1-IQ2_XXS14.9B3.89 GiB0.84 GiB5.58 GiB0.00 GiB50±12.9%
Gemma-4-E4B-it-Minecraft-MT-en-zh-v0.1I1-Q3_K_L8.0B4.68 GiB0.08 GiB5.57 GiB0.01 GiB50±12.9%
gemma-4-E4B-Queen-it-qat-q4_0-unquantizedI1-Q3_K_L8.0B4.68 GiB0.08 GiB5.57 GiB0.01 GiB50±12.9%
Gemma-4-E4B-Luchador-RudoI1-Q3_K_L8.0B4.68 GiB0.08 GiB5.57 GiB0.01 GiB50±12.9%
supergemma4-e4b-abliteratedI1-Q3_K_L7.5B4.68 GiB0.08 GiB5.57 GiB0.01 GiB50±12.9%
Gemma-4-E4B-AbliteratedI1-Q3_K_L8.0B4.68 GiB0.08 GiB5.57 GiB0.01 GiB50±12.9%
gemma-4-E4B-it-The-DECKARD-Claude-Opus-Expresso-Universe-HERETIC-UNCENSORED-ThinkingI1-Q3_K_L8.0B4.68 GiB0.08 GiB5.57 GiB0.01 GiB50±12.9%
gemma-4-E4B-it-The-DECKARD-Expresso-Universe-HERETIC-UNCENSORED-ThinkingI1-Q3_K_L8.0B4.68 GiB0.08 GiB5.57 GiB0.01 GiB50±12.9%
gemma-4-E4B-it-hereticI1-Q3_K_L8.0B4.68 GiB0.08 GiB5.57 GiB0.01 GiB50±12.9%
gemma-4-E4B-it-Claude-Opus-4.5-HERETIC-UNCENSORED-ThinkingI1-Q3_K_L8.0B4.68 GiB0.08 GiB5.57 GiB0.01 GiB50±12.9%
Huihui-gemma-4-E4B-it-abliteratedI1-Q3_K_L8.0B4.68 GiB0.08 GiB5.57 GiB0.01 GiB50±12.9%
gemma-4-E4B-it-Uncensored-MAXI1-Q3_K_L8.0B4.68 GiB0.08 GiB5.57 GiB0.01 GiB50±12.9%
Darkidol-Gemma-4-E4B-itI1-Q3_K_L8.0B4.68 GiB0.08 GiB5.57 GiB0.01 GiB50±12.9%
gemma-4-E4B-it-abliteratedI1-Q3_K_L8.0B4.68 GiB0.08 GiB5.57 GiB0.01 GiB50±12.9%
gemma-4-E4B-it-ultra-uncensored-hereticQ3_K_L8.0B4.68 GiB0.08 GiB5.57 GiB0.01 GiB50±12.9%
OpenMedResearch-Gemma-4E4NI1-Q3_K_L8.0B4.68 GiB0.08 GiB5.57 GiB0.01 GiB50±12.9%
Reasoning-Medical0.1-E4B-sftI1-Q3_K_L8.0B4.68 GiB0.08 GiB5.57 GiB0.01 GiB50±12.9%
gemma-4-E4BQ3_K_L8.0B4.68 GiB0.08 GiB5.57 GiB0.01 GiB50±12.9%
granite-8b-code-instruct-4kIQ4_XS8.1B4.10 GiB0.63 GiB5.57 GiB0.01 GiB50±12.9%
canary-qwen-2.5bBF162.6B4.73 GiB0.00 GiB5.57 GiB0.01 GiB50±12.9%
EXAONE-Deep-7.8BQ4_K_L7.8B4.73 GiB0.00 GiB5.57 GiB0.01 GiB50±12.9%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a GeForce RTX 2060 run?
1188 of 2118 indexed open-weight models fit a GeForce RTX 2060 at 16,384 context with q4_0 KV cache, the largest being gemma-4-12B-it at UD-IQ3_XXS. That covers text, vision-language, image, video and speech models.
How much usable memory does a GeForce RTX 2060 actually have?
Its nameplate is 6 GB, but about 5.58 GiB is available to a model once driver and compositor overhead is accounted for.
Is a GeForce RTX 2060 fast for local AI?
Its memory bandwidth is 336 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.