NVIDIA · workstation

RTX 5000 Ada Generation

RTX 5000 Ada Generation has 32 GB of VRAM at 576 GB/s — about 29.76 GiB usable after driver and compositor overhead. 1994 of 2118 indexed models fit at 64K context with q4_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
32 GB
GDDR6
Bandwidth
576 GB/s
256-bit bus
Tensor FP16
261 TF
dense
TDP
250 W
$4000 MSRP
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
video 16vision language 180text 1710image 2embedding 26audio asr 39audio tts 21

What fits at 64K context

largest quantization that fits, per model · 1994 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
Bernini-RQ8_014.3B28.71 GiB0.00 GiB29.76 GiB0.00 GiB12±22%
Magistral-Small-2509-VisionQ6_K_L24.0B25.83 GiB2.81 GiB29.76 GiB0.00 GiB12±22%
Noromaid-20b-v0.1.1I1-Q2_K20.0B6.91 GiB21.80 GiB29.75 GiB0.01 GiB12±22%
v6-Finch-14B-HFQ6_K_L14.1B11.55 GiB17.16 GiB29.75 GiB0.01 GiB12±22%
Qwen3.5-88BMoEI1-Q2_K_S87.7B28.30 GiB0.42 GiB29.75 GiB0.01 GiB56±37%
Gemma-The-Writer-N-Restless-Quill-10B-UncensoredQ6_K10.0B25.24 GiB3.46 GiB29.75 GiB0.01 GiB12±22%
Open_Gpt4_8x7B_v0.2MoEQ4_K_M46.7B26.43 GiB2.25 GiB29.72 GiB0.04 GiB19±37%
GPT-NeoX-20B-ErebusI1-Q3_K_M20.6B10.03 GiB18.56 GiB29.68 GiB0.08 GiB12±22%
Skyfall-31B-v4.2Q6_K_L31.4B24.74 GiB3.80 GiB29.65 GiB0.11 GiB12±22%
Delphi-25B-SimpleRL-MathI1-IQ3_XS25.0B9.74 GiB18.83 GiB29.64 GiB0.12 GiB12±22%
Qwen3.6-27B-Fable-5-ExperimentalQ8_027.8B27.42 GiB1.13 GiB29.61 GiB0.15 GiB12±22%
Qwen3-53B-A3B-2507-THINKING-TOTAL-RECALL-v2-MASTER-CODERMoEI1-Q3_K_L53.0B25.64 GiB2.95 GiB29.59 GiB0.17 GiB29±37%
Darwin-35B-A3B-OpusMoEQ6_K_L36.0B28.22 GiB0.35 GiB29.57 GiB0.19 GiB63±37%
Aurora-Code-1MoEQ6_K_L34.7B28.22 GiB0.35 GiB29.57 GiB0.19 GiB63±37%
grug-35b-v2MoEQ6_K_L35.1B28.22 GiB0.35 GiB29.57 GiB0.19 GiB63±37%
grug-35bMoEQ6_K_L35.1B28.22 GiB0.35 GiB29.57 GiB0.19 GiB63±37%
WorldSim-Opus-3.6-35B-A3BMoEQ6_K_L35.1B28.22 GiB0.35 GiB29.57 GiB0.19 GiB63±37%
Qwen3.6-35B-A3B-AnkoMoEQ6_K_L35.1B28.22 GiB0.35 GiB29.57 GiB0.19 GiB63±37%
KAT-Coder-V2.5-DevMoEQ6_K_L34.7B28.22 GiB0.35 GiB29.57 GiB0.19 GiB63±37%
Ornith-1.0-35BMoEQ6_K_L34.7B28.22 GiB0.35 GiB29.57 GiB0.19 GiB63±37%
Nex-N2-miniMoEQ6_K_L35.1B28.22 GiB0.35 GiB29.57 GiB0.19 GiB63±37%
Maenad-70BI1-Q2_K_S70.6B22.79 GiB5.63 GiB29.54 GiB0.22 GiB12±22%
DeepSeek-R1-Distill-Llama-70B-Uncensored-v2-Unbiased-ReasonerI1-Q2_K_S70.6B22.79 GiB5.63 GiB29.54 GiB0.22 GiB12±22%
Rombos-LLM-70b-Llama-3.3I1-Q2_K_S70.6B22.79 GiB5.63 GiB29.54 GiB0.22 GiB12±22%
L3.3-Electra-R1-70bI1-Q2_K_S70.6B22.79 GiB5.63 GiB29.54 GiB0.22 GiB12±22%
Latxa-Llama-3.1-70B-Instruct-v2I1-Q2_K_S70.6B22.79 GiB5.63 GiB29.54 GiB0.22 GiB12±22%
Llama-3.3_70_b_uncensored_continuedI1-Q2_K_S70.6B22.79 GiB5.63 GiB29.54 GiB0.22 GiB12±22%
Llama-3.3-70B-Instruct-abliteratedI1-Q2_K_S70.6B22.79 GiB5.63 GiB29.54 GiB0.22 GiB12±22%
grok-oss-Revenant-70BI1-Q2_K_S70.6B22.79 GiB5.63 GiB29.54 GiB0.22 GiB12±22%
Llama-3.1-Nemotron-70B-Instruct-HFI1-Q2_K_S70.6B22.79 GiB5.63 GiB29.54 GiB0.22 GiB12±22%
L3.3-70B-Euryale-v2.3I1-Q2_K_S70.6B22.79 GiB5.63 GiB29.54 GiB0.22 GiB12±22%
Hermes-4-70B-hereticI1-Q2_K_S70.6B22.79 GiB5.63 GiB29.54 GiB0.22 GiB12±22%
Llama-3.1-70BQ2_K_S70.6B22.79 GiB5.63 GiB29.54 GiB0.22 GiB12±22%
Golem-70B-v1bI1-Q2_K_S70.6B22.79 GiB5.63 GiB29.54 GiB0.22 GiB12±22%
DeepSeek-R1-Distill-Llama-70B-abliteratedI1-Q2_K_S70.6B22.79 GiB5.63 GiB29.54 GiB0.22 GiB12±22%
DeepSeek-R1-Distill-Llama-70B-hereticI1-Q2_K_S70.6B22.79 GiB5.63 GiB29.54 GiB0.22 GiB12±22%
Legion-V2.1-LLaMa-70BI1-Q2_K_S70.6B22.79 GiB5.63 GiB29.54 GiB0.22 GiB12±22%
Assistant_Pepe_70BI1-Q2_K_S70.6B22.79 GiB5.63 GiB29.54 GiB0.22 GiB12±22%
Athene-70BQ2_K_S70.6B22.79 GiB5.63 GiB29.54 GiB0.22 GiB12±22%
Hermes-3-Llama-3.1-70BQ2_K_S70.6B22.79 GiB5.63 GiB29.54 GiB0.22 GiB12±22%
L3.3-70B-Magnum-DiamondQ2_K_S70.6B22.79 GiB5.63 GiB29.54 GiB0.22 GiB12±22%
Meta-Llama-3-70B-Instruct-abliterated-v3.5Q2_K_S70.6B22.79 GiB5.63 GiB29.54 GiB0.22 GiB12±22%
Kimi-Linear-48B-A3B-InstructMoEQ4_K_M49.1B28.00 GiB0.53 GiB29.54 GiB0.22 GiB12±22%
Hypernova-60B-2605MoEI1-IQ3_XXS58.7B27.90 GiB0.57 GiB29.46 GiB0.30 GiB52±37%
DeepSeek-R1-Distill-Llama-70BUD-IQ2_M70.6B22.70 GiB5.63 GiB29.45 GiB0.31 GiB12±22%
Seed-OSS-36B-Instruct-biprojected-norm-preserving-abliteratedI1-Q5_K_M36.2B23.84 GiB4.50 GiB29.44 GiB0.32 GiB12±22%
Seed-OSS-36B-InstructQ5_K_M36.2B23.84 GiB4.50 GiB29.44 GiB0.32 GiB12±22%
Hermes-4.3-36B-hereticI1-Q5_K_M36.2B23.84 GiB4.50 GiB29.44 GiB0.32 GiB12±22%
Hermes-4.3-36BQ5_K_M36.2B23.84 GiB4.50 GiB29.44 GiB0.32 GiB12±22%
Seed-OSS-36B-BaseQ5_K_M36.2B23.84 GiB4.50 GiB29.44 GiB0.32 GiB12±22%
Gemma4-Gutenberg-31BQ6_K_L31.3B25.21 GiB3.14 GiB29.43 GiB0.33 GiB12±22%
gemma-4-31B-itQ6_K_L31.3B25.21 GiB3.14 GiB29.43 GiB0.33 GiB12±22%
Gemma4-Gutenberg-31B-HereticQ6_K_L31.3B25.21 GiB3.14 GiB29.43 GiB0.33 GiB12±22%
Equinox-31BQ6_K_L31.3B25.21 GiB3.14 GiB29.43 GiB0.33 GiB12±22%
gemma-4-31B-it-SDFT-Heretic-RPQ6_K_L30.7B25.21 GiB3.14 GiB29.43 GiB0.33 GiB12±22%
Apertus-70B-Instruct-2509IQ2_M70.6B22.61 GiB5.63 GiB29.41 GiB0.35 GiB12±22%
medgemma-27b-itQ8_028.8B26.74 GiB1.58 GiB29.39 GiB0.37 GiB12±22%
gemma-3-27b-it-abliterated-refined-visionQ8_027.4B26.74 GiB1.58 GiB29.39 GiB0.37 GiB12±22%
gemma-3-27b-it-abliteratedQ8_027.4B26.74 GiB1.58 GiB29.39 GiB0.37 GiB12±22%
Nidum-Gemma-3-27B-it-UncensoredQ8_027.4B26.74 GiB1.58 GiB29.39 GiB0.37 GiB12±22%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a RTX 5000 Ada Generation run?
1994 of 2118 indexed open-weight models fit a RTX 5000 Ada Generation at 65,536 context with q4_0 KV cache, the largest being Bernini-R at Q8_0. That covers text, vision-language, image, video and speech models.
How much usable memory does a RTX 5000 Ada Generation actually have?
Its nameplate is 32 GB, but about 29.76 GiB is available to a model once driver and compositor overhead is accounted for.
Is a RTX 5000 Ada Generation fast for local AI?
Its memory bandwidth is 576 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.
RTX 5000 Ada Generation — what AI models can it run locally? — ossmodeldb