NVIDIA · consumer

GeForce RTX 3060

GeForce RTX 3060 has 8 GB of VRAM at 240 GB/s — about 7.44 GiB usable after driver and compositor overhead. 929 of 2118 indexed models fit at 64K context with q8_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
8 GB
GDDR6
Bandwidth
240 GB/s
128-bit bus
Tensor FP16
51 TF
dense
TDP
195 W
$329 MSRP
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 751embedding 25vision language 86audio tts 20audio asr 38video 8image 1

What fits at 64K context

largest quantization that fits, per model · 929 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
CycleGRPO-4BI1-IQ3_XS4.8B1.85 GiB4.78 GiB7.44 GiB0.00 GiB26±12.9%
Jan-v3-4B-base-instructIQ3_XS4.4B1.85 GiB4.78 GiB7.44 GiB0.00 GiB26±12.9%
Jan-code-4bIQ3_XS4.4B1.85 GiB4.78 GiB7.44 GiB0.00 GiB26±12.9%
OmniAtlas-Qwen3-30B-A3BI1-IQ1_M31.7B6.59 GiB0.00 GiB7.44 GiB0.00 GiB26±12.9%
Qwen3-Omni-30B-A3B-CaptionerI1-IQ1_M31.7B6.59 GiB0.00 GiB7.44 GiB0.00 GiB26±12.9%
nomic-embed-codeQ5_K_M7.1B4.72 GiB1.86 GiB7.44 GiB0.00 GiB26±12.9%
Fara1.5-9BQ4_19.4B5.54 GiB1.06 GiB7.43 GiB0.01 GiB26±12.9%
QwenPaw-Flash-9BQ4_19.4B5.54 GiB1.06 GiB7.43 GiB0.01 GiB26±12.9%
grug-9bQ4_19.4B5.54 GiB1.06 GiB7.43 GiB0.01 GiB26±12.9%
OmniCoder-9BQ4_19.4B5.54 GiB1.06 GiB7.43 GiB0.01 GiB26±12.9%
Ornith-1.0-9BQ4_19.2B5.54 GiB1.06 GiB7.43 GiB0.01 GiB26±12.9%
Qwen3.5-9B-NeoQ4_19.7B5.54 GiB1.06 GiB7.43 GiB0.01 GiB26±12.9%
Qwythos-9B-Claude-Mythos-5-1M-uncensored-hereticQ4_K_S9.4B5.53 GiB1.06 GiB7.43 GiB0.01 GiB26±12.9%
rnj-1-instructUD-IQ2_XXS8.3B2.33 GiB4.25 GiB7.43 GiB0.01 GiB26±12.9%
zeta-2.1I1-IQ2_XXS8.3B2.34 GiB4.25 GiB7.43 GiB0.01 GiB26±12.9%
orpheus-3b-0.1-pretrainedQ6_K3.8B2.90 GiB3.72 GiB7.43 GiB0.01 GiB26±12.9%
Hubble-4B-v1IQ4_XS4.5B2.36 GiB4.25 GiB7.42 GiB0.02 GiB26±12.9%
Aura-4BI1-IQ4_XS4.5B2.36 GiB4.25 GiB7.42 GiB0.02 GiB26±12.9%
magnum-v2-4bI1-IQ4_XS4.5B2.36 GiB4.25 GiB7.42 GiB0.02 GiB26±12.9%
Impish_LLAMA_4BIQ4_XS4.5B2.36 GiB4.25 GiB7.42 GiB0.02 GiB26±12.9%
Llama-3.1-8B-InstructUD-IQ2_XXS8.0B2.33 GiB4.25 GiB7.42 GiB0.02 GiB26±12.9%
Llama-3.1-Nemotron-Nano-8B-v1UD-IQ2_XXS8.0B2.33 GiB4.25 GiB7.42 GiB0.02 GiB26±12.9%
DeepSeek-R1-Distill-Llama-8BUD-IQ2_XXS8.0B2.33 GiB4.25 GiB7.42 GiB0.02 GiB26±12.9%
Gemma-4-12B-StyleTuneI1-IQ2_S13.0B4.20 GiB2.37 GiB7.42 GiB0.02 GiB26±12.9%
gemma-4-12b-heretic-styletune-headI1-IQ2_S12.0B4.20 GiB2.37 GiB7.42 GiB0.02 GiB26±12.9%
syrian-gemma-12bI1-IQ2_S13.0B4.20 GiB2.37 GiB7.42 GiB0.02 GiB26±12.9%
Voxtral-Mini-3B-2507Q5_K_S4.7B2.63 GiB3.98 GiB7.42 GiB0.02 GiB26±12.9%
Qwen3-VL-4B-Instruct-Unredacted-MAXI1-IQ3_M4.4B1.83 GiB4.78 GiB7.42 GiB0.02 GiB26±12.9%
Qwen3-VL-4B-Thinking-Unredacted-MAXI1-IQ3_M4.4B1.83 GiB4.78 GiB7.42 GiB0.02 GiB26±12.9%
Zubr1.0-VL-4BI1-IQ3_M4.4B1.83 GiB4.78 GiB7.42 GiB0.02 GiB26±12.9%
Huihui-Qwen3-VL-4B-Instruct-abliteratedI1-IQ3_M4.4B1.83 GiB4.78 GiB7.42 GiB0.02 GiB26±12.9%
Qwen3-VL-4B-Instruct-UncensoredI1-IQ3_M4.4B1.83 GiB4.78 GiB7.42 GiB0.02 GiB26±12.9%
OpenCaption-4B-VL-SFT-v1.0I1-IQ3_M4.4B1.83 GiB4.78 GiB7.42 GiB0.02 GiB26±12.9%
Parable-Qwen3-4B-Claude-Fable-5I1-IQ3_M4.0B1.83 GiB4.78 GiB7.42 GiB0.02 GiB26±12.9%
Jan-v1-4BIQ3_M4.0B1.83 GiB4.78 GiB7.42 GiB0.02 GiB26±12.9%
Qwen3-4b-Z-Image-Turbo-AbliteratedV1I1-IQ3_M4.0B1.83 GiB4.78 GiB7.42 GiB0.02 GiB26±12.9%
Jan-nano-128kIQ3_M4.0B1.83 GiB4.78 GiB7.42 GiB0.02 GiB26±12.9%
Qwen3-4B-Instruct-2507IQ3_M4.0B1.83 GiB4.78 GiB7.42 GiB0.02 GiB26±12.9%
Qwen3-4B-Thinking-2507IQ3_M4.0B1.83 GiB4.78 GiB7.42 GiB0.02 GiB26±12.9%
Jan-nanoIQ3_M4.0B1.83 GiB4.78 GiB7.42 GiB0.02 GiB26±12.9%
Qwen3-4B-abliteratedIQ3_M4.0B1.83 GiB4.78 GiB7.42 GiB0.02 GiB26±12.9%
Qwen3-4B-BaseIQ3_M4.0B1.83 GiB4.78 GiB7.42 GiB0.02 GiB26±12.9%
Neuron-4B-InstructI1-IQ3_M4.0B1.83 GiB4.78 GiB7.42 GiB0.02 GiB26±12.9%
ChineseErrorCorrector4-4BI1-IQ3_M4.0B1.83 GiB4.78 GiB7.42 GiB0.02 GiB26±12.9%
FastContext-1.0-4B-SFT-abliteratedI1-IQ3_M4.0B1.83 GiB4.78 GiB7.42 GiB0.02 GiB26±12.9%
Qwen3-4B-Instruct_NSFW-V2.1I1-IQ3_M4.0B1.83 GiB4.78 GiB7.42 GiB0.02 GiB26±12.9%
FastContext-1.0-4B-SFTI1-IQ3_M4.0B1.83 GiB4.78 GiB7.42 GiB0.02 GiB26±12.9%
fable-traces-abliteratedI1-IQ3_M4.0B1.83 GiB4.78 GiB7.42 GiB0.02 GiB26±12.9%
Nexa-AI-4B-InstructI1-IQ3_M4.0B1.83 GiB4.78 GiB7.42 GiB0.02 GiB26±12.9%
Lumen-4B-InstructI1-IQ3_M4.0B1.83 GiB4.78 GiB7.42 GiB0.02 GiB26±12.9%
fable-tracesIQ3_M4.0B1.83 GiB4.78 GiB7.42 GiB0.02 GiB26±12.9%
Qwen3-4B-Instruct-2507-hereticIQ3_M4.0B1.83 GiB4.78 GiB7.42 GiB0.02 GiB26±12.9%
Qwen3-HereticLM-4BI1-IQ3_M4.0B1.83 GiB4.78 GiB7.42 GiB0.02 GiB26±12.9%
dolphin-2.9.3-mistral-7B-32kI1-IQ2_M7.2B2.33 GiB4.25 GiB7.42 GiB0.02 GiB26±12.9%
Mistral-7B-v0.3IQ2_M7.2B2.33 GiB4.25 GiB7.42 GiB0.02 GiB26±12.9%
Mistral-7B-Instruct-v0.3-ParasiteI1-IQ2_M7.2B2.33 GiB4.25 GiB7.42 GiB0.02 GiB26±12.9%
Mistral-7B-Instruct-v0.3-JbliteratedI1-IQ2_M7.2B2.33 GiB4.25 GiB7.42 GiB0.02 GiB26±12.9%
Mathstral-7B-v0.1IQ2_M7.2B2.33 GiB4.25 GiB7.42 GiB0.02 GiB26±12.9%
SciPhi-Self-RAG-Mistral-7B-32kKV unresolvedI1-IQ2_M7.2B2.33 GiB4.25 GiB7.42 GiB0.02 GiB26±12.9%
dolphin-2.2.1-mistral-7bKV unresolvedI1-IQ2_M7.2B2.33 GiB4.25 GiB7.42 GiB0.02 GiB26±12.9%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a GeForce RTX 3060 run?
929 of 2118 indexed open-weight models fit a GeForce RTX 3060 at 65,536 context with q8_0 KV cache, the largest being CycleGRPO-4B at I1-IQ3_XS. That covers text, vision-language, image, video and speech models.
How much usable memory does a GeForce RTX 3060 actually have?
Its nameplate is 8 GB, but about 7.44 GiB is available to a model once driver and compositor overhead is accounted for.
Is a GeForce RTX 3060 fast for local AI?
Its memory bandwidth is 240 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.