NVIDIA · consumer

GeForce RTX 4070 GDDR6

GeForce RTX 4070 GDDR6 has 12 GB of VRAM at 480 GB/s — about 11.16 GiB usable after driver and compositor overhead. 876 of 2118 indexed models fit at 128K context with q8_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
12 GB
GDDR6
Bandwidth
480 GB/s
192-bit bus
Tensor FP16
117 TF
dense
TDP
200 W
$599 MSRP
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
video 14vision language 95audio tts 20text 687audio asr 38embedding 21image 1

What fits at 128K context

largest quantization that fits, per model · 876 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
Wan2.1-FLF2V-14B-720PQ4_116.4B10.32 GiB0.00 GiB11.16 GiB0.00 GiB33±12.9%
Wan2.1-I2V-14B-480PQ4_116.4B10.32 GiB0.00 GiB11.15 GiB0.01 GiB33±12.9%
Wan2.1-I2V-14B-720PQ4_116.4B10.32 GiB0.00 GiB11.15 GiB0.01 GiB33±12.9%
Muse-Glimmer-30BIQ2_S29.8B9.35 GiB0.91 GiB11.15 GiB0.01 GiB34±12.9%
orpheus-3b-0.1-pretrainedQ6_K3.8B2.90 GiB7.44 GiB11.15 GiB0.01 GiB33±12.9%
DeepSeek-Coder-V2-Lite-BaseMoEI1-Q4_015.7B8.32 GiB2.02 GiB11.14 GiB0.02 GiB59±37%
Voxtral-Mini-4B-Realtime-2602KV unresolvedQ6_K4.4B3.41 GiB6.91 GiB11.13 GiB0.03 GiB33±12.9%
DeepSeek-Coder-V2-Lite-InstructMoEIQ4_NL15.7B8.29 GiB2.02 GiB11.12 GiB0.04 GiB59±37%
DeepSeek-V2-Lite-ChatMoEIQ4_NL15.7B8.29 GiB2.02 GiB11.12 GiB0.04 GiB59±37%
Ministral-3-3B-Instruct-2512Q8_03.8B3.40 GiB6.91 GiB11.12 GiB0.04 GiB33±12.9%
Ministral-3-3B-Reasoning-2512Q8_04.3B3.40 GiB6.91 GiB11.12 GiB0.04 GiB33±12.9%
Ministral-3-3B-Instruct-2512-BF16Q8_04.3B3.40 GiB6.91 GiB11.12 GiB0.04 GiB33±12.9%
Amaretto-3BQ8_04.3B3.40 GiB6.91 GiB11.12 GiB0.04 GiB33±12.9%
ERNIE-4.5-21B-A3B-PTUD-IQ1_S21.9B6.58 GiB3.72 GiB11.12 GiB0.04 GiB33±12.9%
Marco-Nano-InstructMoEI1-IQ2_S8.0B2.91 GiB7.44 GiB11.12 GiB0.04 GiB26±37%
Wan2.2-Distill-ModelsQ5_114.3B10.27 GiB0.00 GiB11.10 GiB0.06 GiB33±12.9%
Bernini-RQ5_114.3B10.26 GiB0.00 GiB11.10 GiB0.06 GiB34±12.9%
SkyReels-V2-DF-14B-540PQ5_114.3B10.27 GiB0.00 GiB11.10 GiB0.06 GiB34±12.9%
Gemma-4-12B-StyleTuneI1-IQ3_M13.0B5.74 GiB4.50 GiB11.09 GiB0.07 GiB34±12.9%
gemma-4-12b-heretic-styletune-headI1-IQ3_M12.0B5.74 GiB4.50 GiB11.09 GiB0.07 GiB34±12.9%
syrian-gemma-12bI1-IQ3_M13.0B5.74 GiB4.50 GiB11.09 GiB0.07 GiB34±12.9%
InternVL3_5-14BQ5_K_L15.1B10.24 GiB0.00 GiB11.08 GiB0.08 GiB34±12.9%
Phi-4-mini-reasoningQ3_K_S3.8B1.77 GiB8.50 GiB11.08 GiB0.08 GiB34±12.9%
Phi-4-mini-instructQ3_K_S3.8B1.77 GiB8.50 GiB11.08 GiB0.08 GiB34±12.9%
Yi-1.5-6B-ChatQ8_06.1B6.00 GiB4.25 GiB11.07 GiB0.09 GiB34±12.9%
Trinity-MiniMoEQ2_K_L26.1B9.15 GiB1.12 GiB11.07 GiB0.09 GiB85±37%
ERNIE-4.5-21B-A3B-ThinkingUD-IQ1_S21.8B6.53 GiB3.72 GiB11.07 GiB0.09 GiB34±12.9%
Aura-4BI1-IQ3_XXS4.5B1.75 GiB8.50 GiB11.06 GiB0.10 GiB34±12.9%
magnum-v2-4bI1-IQ3_XXS4.5B1.75 GiB8.50 GiB11.06 GiB0.10 GiB34±12.9%
Impish_LLAMA_4BIQ3_XXS4.5B1.75 GiB8.50 GiB11.06 GiB0.10 GiB34±12.9%
granite-20b-code-instruct-8kIQ4_XS20.1B10.19 GiB0.00 GiB11.06 GiB0.10 GiB34±12.9%
granite-20b-code-base-8kI1-IQ4_XS20.1B10.19 GiB0.00 GiB11.06 GiB0.10 GiB34±12.9%
Kimi-VL-A3B-Thinking-2506MoEIQ4_XS16.4B8.21 GiB2.02 GiB11.05 GiB0.11 GiB59±37%
Kimi-VL-A3B-InstructMoEIQ4_XS16.4B8.21 GiB2.02 GiB11.05 GiB0.11 GiB59±37%
Qwen3.6-28BMoEI1-IQ2_M28.2B8.91 GiB1.33 GiB11.04 GiB0.12 GiB86±37%
Qwen3.5-28BMoEI1-IQ2_M28.7B8.91 GiB1.33 GiB11.04 GiB0.12 GiB86±37%
GLM-4.7-Flash-REAP-23B-A3BMoEUD-IQ1_S23.0B6.71 GiB3.51 GiB11.03 GiB0.13 GiB44±37%
OLMoE-1B-7B-0924-InstructMoEI1-IQ2_XXS6.9B1.76 GiB8.50 GiB11.03 GiB0.13 GiB22±37%
Llama-3.1-Minitron-4B-Width-BaseQ2_K4.5B1.71 GiB8.50 GiB11.03 GiB0.13 GiB34±12.9%
Qwen3.6-35B-A3B-REAM-160-ru-agentMoEIQ3_XXS23.6B8.89 GiB1.33 GiB11.02 GiB0.14 GiB83±37%
Tiger-Gemma-12B-v3IQ3_M12.8B5.67 GiB4.50 GiB11.01 GiB0.15 GiB34±12.9%
AfriqueGemma-12BI1-IQ3_M12.2B5.67 GiB4.50 GiB11.01 GiB0.15 GiB34±12.9%
SuperGemma-4-12b-abliteratedI1-Q3_K_M12.0B5.67 GiB4.50 GiB11.01 GiB0.15 GiB34±12.9%
gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-uncensored-hereticI1-Q3_K_M12.0B5.67 GiB4.50 GiB11.01 GiB0.15 GiB34±12.9%
gemma-4-12B-coder-fable5-composer2.5-v1-uncensored-hereticI1-Q3_K_M12.0B5.67 GiB4.50 GiB11.01 GiB0.15 GiB34±12.9%
gemma-4-12B-it-uncensored-hereticI1-Q3_K_M12.0B5.67 GiB4.50 GiB11.01 GiB0.15 GiB34±12.9%
gemma-4-12B-it-qat-q4_0-unquantized-uncensored-hereticQ3_K_M12.0B5.67 GiB4.50 GiB11.01 GiB0.15 GiB34±12.9%
Grug-12BI1-Q3_K_M12.0B5.67 GiB4.50 GiB11.01 GiB0.15 GiB34±12.9%
Aura-Medium-v1-BF16I1-Q3_K_M12.0B5.67 GiB4.50 GiB11.01 GiB0.15 GiB34±12.9%
gemma-4-12B-it-Esper4I1-Q3_K_M12.0B5.67 GiB4.50 GiB11.01 GiB0.15 GiB34±12.9%
gemma-4-12B-it-GuardpointI1-Q3_K_M12.0B5.67 GiB4.50 GiB11.01 GiB0.15 GiB34±12.9%
Gemma-4-12B-it-AEON-Abliterated-K4-BF16I1-Q3_K_M12.0B5.67 GiB4.50 GiB11.01 GiB0.15 GiB34±12.9%
gemma-4-12B-it-Tachibana-AgentI1-Q3_K_M12.0B5.67 GiB4.50 GiB11.01 GiB0.15 GiB34±12.9%
gemma-4-12b-marvin-gutenberg-rp-v2I1-Q3_K_M12.0B5.67 GiB4.50 GiB11.01 GiB0.15 GiB34±12.9%
gemma-4-12b-crownelius-writerI1-Q3_K_M12.0B5.67 GiB4.50 GiB11.01 GiB0.15 GiB34±12.9%
Huihui-gemma-4-12B-coder-fable5-composer2.5-v1-abliteratedI1-Q3_K_M12.0B5.67 GiB4.50 GiB11.01 GiB0.15 GiB34±12.9%
gemma-4-12b-asterion-agenticI1-Q3_K_M12.0B5.67 GiB4.50 GiB11.01 GiB0.15 GiB34±12.9%
Huihui-gemma-4-12B-agentic-fable5-abliteratedI1-Q3_K_M12.0B5.67 GiB4.50 GiB11.01 GiB0.15 GiB34±12.9%
g4-12b-it-trismegistusI1-Q3_K_M12.0B5.67 GiB4.50 GiB11.01 GiB0.15 GiB34±12.9%
gemma4-12b-it-asimovI1-Q3_K_M12.0B5.67 GiB4.50 GiB11.01 GiB0.15 GiB34±12.9%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a GeForce RTX 4070 GDDR6 run?
876 of 2118 indexed open-weight models fit a GeForce RTX 4070 GDDR6 at 131,072 context with q8_0 KV cache, the largest being Wan2.1-FLF2V-14B-720P at Q4_1. That covers text, vision-language, image, video and speech models.
How much usable memory does a GeForce RTX 4070 GDDR6 actually have?
Its nameplate is 12 GB, but about 11.16 GiB is available to a model once driver and compositor overhead is accounted for.
Is a GeForce RTX 4070 GDDR6 fast for local AI?
Its memory bandwidth is 480 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.