NVIDIA · datacenter

Tesla V100 32GB

Tesla V100 32GB has 32 GB of VRAM at 900 GB/s — about 29.76 GiB usable after driver and compositor overhead. 1996 of 2118 indexed models fit at 32K context with q8_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
32 GB
HBM2
Bandwidth
900 GB/s
4096-bit bus
Tensor FP16
125 TF
dense
TDP
300 W
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
video 16text 1712vision language 180image 2embedding 26audio asr 39audio tts 21

What fits at 32K context

largest quantization that fits, per model · 1996 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
Bernini-RQ8_014.3B28.71 GiB0.00 GiB29.76 GiB0.00 GiB18±22%
Qwen3-Next-80B-A3B-ThinkingMoEQ2_K81.3B27.17 GiB1.59 GiB29.75 GiB0.01 GiB76±37%
Qwen3-Next-80B-A3B-InstructMoEQ2_K81.3B27.17 GiB1.59 GiB29.75 GiB0.01 GiB76±37%
xLAM-8x7b-rMoEQ4_K_L46.7B26.59 GiB2.13 GiB29.75 GiB0.01 GiB29±37%
Qwen3.5-88BMoEI1-Q2_K_S87.7B28.30 GiB0.40 GiB29.72 GiB0.04 GiB86±37%
Noromaid-20b-v0.1.1I1-IQ3_S20.0B8.06 GiB20.59 GiB29.69 GiB0.07 GiB18±22%
Nethena-20BQ3_K_S20.0B8.06 GiB20.59 GiB29.69 GiB0.07 GiB18±22%
dolphin-2.6-mixtral-8x7bMoEI1-Q4_K_M46.7B26.49 GiB2.13 GiB29.66 GiB0.10 GiB29±37%
Nous-Hermes-2-Mixtral-8x7B-DPOMoEQ4_K_M46.7B26.49 GiB2.13 GiB29.66 GiB0.10 GiB29±37%
Mixtral-8x7B-Instruct-v0.1MoEQ4_K_M46.7B26.49 GiB2.13 GiB29.66 GiB0.10 GiB29±37%
dolphin-2.5-mixtral-8x7bMoEQ4_K_M46.7B26.49 GiB2.13 GiB29.65 GiB0.11 GiB29±37%
Mixtral-8x7B-v0.1MoEQ4_K_M46.7B26.49 GiB2.13 GiB29.65 GiB0.11 GiB29±37%
IQuest-Coder-V1-40B-InstructI1-Q4_139.8B23.24 GiB5.31 GiB29.65 GiB0.11 GiB18±22%
Seed-OSS-36B-InstructQ5_K_L36.2B24.29 GiB4.25 GiB29.64 GiB0.12 GiB18±22%
Hermes-4.3-36BQ5_K_L36.2B24.29 GiB4.25 GiB29.64 GiB0.12 GiB18±22%
Apertus-70B-Instruct-2509UD-IQ2_M70.6B23.12 GiB5.31 GiB29.61 GiB0.15 GiB18±22%
Magistral-Small-2509-VisionQ6_K_L24.0B25.83 GiB2.66 GiB29.60 GiB0.16 GiB18±22%
Open_Gpt4_8x7B_v0.2MoEQ4_K_M46.7B26.43 GiB2.13 GiB29.59 GiB0.17 GiB29±37%
GPT-NeoX-20B-ErebusI1-Q3_K_L20.6B10.96 GiB17.53 GiB29.59 GiB0.17 GiB18±22%
Gemma4-Gutenberg-31BQ6_K_L31.3B25.21 GiB3.28 GiB29.57 GiB0.19 GiB18±22%
gemma-4-31B-itQ6_K_L31.3B25.21 GiB3.28 GiB29.57 GiB0.19 GiB18±22%
Gemma4-Gutenberg-31B-HereticQ6_K_L31.3B25.21 GiB3.28 GiB29.57 GiB0.19 GiB18±22%
Equinox-31BQ6_K_L31.3B25.21 GiB3.28 GiB29.57 GiB0.19 GiB18±22%
gemma-4-31B-it-SDFT-Heretic-RPQ6_K_L30.7B25.21 GiB3.28 GiB29.57 GiB0.19 GiB18±22%
Delphi-25B-SimpleRL-MathI1-IQ3_M25.0B10.70 GiB17.78 GiB29.56 GiB0.20 GiB18±22%
Darwin-35B-A3B-OpusMoEQ6_K_L36.0B28.22 GiB0.33 GiB29.56 GiB0.20 GiB98±37%
Aurora-Code-1MoEQ6_K_L34.7B28.22 GiB0.33 GiB29.56 GiB0.20 GiB98±37%
grug-35b-v2MoEQ6_K_L35.1B28.22 GiB0.33 GiB29.56 GiB0.20 GiB98±37%
grug-35bMoEQ6_K_L35.1B28.22 GiB0.33 GiB29.56 GiB0.20 GiB98±37%
WorldSim-Opus-3.6-35B-A3BMoEQ6_K_L35.1B28.22 GiB0.33 GiB29.56 GiB0.20 GiB98±37%
Qwen3.6-35B-A3B-AnkoMoEQ6_K_L35.1B28.22 GiB0.33 GiB29.56 GiB0.20 GiB98±37%
KAT-Coder-V2.5-DevMoEQ6_K_L34.7B28.22 GiB0.33 GiB29.56 GiB0.20 GiB98±37%
Ornith-1.0-35BMoEQ6_K_L34.7B28.22 GiB0.33 GiB29.56 GiB0.20 GiB98±37%
Nex-N2-miniMoEQ6_K_L35.1B28.22 GiB0.33 GiB29.56 GiB0.20 GiB98±37%
Qwen3.6-27B-Fable-5-ExperimentalQ8_027.8B27.42 GiB1.06 GiB29.55 GiB0.21 GiB18±22%
Kimi-Linear-48B-A3B-InstructMoEQ4_K_M49.1B28.00 GiB0.50 GiB29.51 GiB0.25 GiB18±22%
medgemma-27b-itQ8_028.8B26.74 GiB1.65 GiB29.47 GiB0.29 GiB18±22%
gemma-3-27b-it-abliterated-refined-visionQ8_027.4B26.74 GiB1.65 GiB29.47 GiB0.29 GiB18±22%
gemma-3-27b-it-abliteratedQ8_027.4B26.74 GiB1.65 GiB29.47 GiB0.29 GiB18±22%
Nidum-Gemma-3-27B-it-UncensoredQ8_027.4B26.74 GiB1.65 GiB29.47 GiB0.29 GiB18±22%
gemma-3-27b-itQ8_027.4B26.74 GiB1.65 GiB29.47 GiB0.29 GiB18±22%
Unbound-v1.12.0-27BQ8_027.4B26.74 GiB1.65 GiB29.47 GiB0.29 GiB18±22%
medgemma-27b-text-itQ8_027.0B26.74 GiB1.65 GiB29.47 GiB0.29 GiB18±22%
CalmeRys-78B-Orpo-v0.1I1-IQ1_S78.0B22.62 GiB5.71 GiB29.47 GiB0.29 GiB18±22%
Skyfall-31B-v4.2Q6_K_L31.4B24.74 GiB3.59 GiB29.44 GiB0.32 GiB18±22%
Hypernova-60B-2605MoEI1-IQ3_XXS58.7B27.90 GiB0.54 GiB29.43 GiB0.33 GiB80±37%
Qwen3-53B-A3B-2507-THINKING-TOTAL-RECALL-v2-MASTER-CODERMoEI1-Q3_K_L53.0B25.64 GiB2.79 GiB29.42 GiB0.34 GiB46±37%
glm-4-9b-chat-1mF169.5B17.67 GiB10.63 GiB29.34 GiB0.42 GiB18±22%
Qwen3.6-28BMoEQ8_028.2B28.00 GiB0.33 GiB29.34 GiB0.42 GiB92±37%
gemma-4-E4B-it-Uncensored-MAXF328.0B28.02 GiB0.27 GiB29.31 GiB0.45 GiB18±22%
Qwen3.6-35B-A3BMoEUD-Q6_K36.0B27.95 GiB0.33 GiB29.29 GiB0.47 GiB98±37%
Pantheon-Reasoning-27BQ8_027.8B27.12 GiB1.06 GiB29.24 GiB0.52 GiB18±22%
ThinkingCap-Qwen3.6-27BQ8_027.4B27.12 GiB1.06 GiB29.24 GiB0.52 GiB18±22%
Qwen3.5-27BQ8_027.8B27.12 GiB1.06 GiB29.24 GiB0.52 GiB18±22%
Maenad-70BI1-Q2_K_S70.6B22.79 GiB5.31 GiB29.23 GiB0.53 GiB18±22%
DeepSeek-R1-Distill-Llama-70B-Uncensored-v2-Unbiased-ReasonerI1-Q2_K_S70.6B22.79 GiB5.31 GiB29.23 GiB0.53 GiB18±22%
Rombos-LLM-70b-Llama-3.3I1-Q2_K_S70.6B22.79 GiB5.31 GiB29.23 GiB0.53 GiB18±22%
L3.3-Electra-R1-70bI1-Q2_K_S70.6B22.79 GiB5.31 GiB29.23 GiB0.53 GiB18±22%
Latxa-Llama-3.1-70B-Instruct-v2I1-Q2_K_S70.6B22.79 GiB5.31 GiB29.23 GiB0.53 GiB18±22%
Llama-3.3_70_b_uncensored_continuedI1-Q2_K_S70.6B22.79 GiB5.31 GiB29.23 GiB0.53 GiB18±22%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a Tesla V100 32GB run?
1996 of 2118 indexed open-weight models fit a Tesla V100 32GB at 32,768 context with q8_0 KV cache, the largest being Bernini-R at Q8_0. That covers text, vision-language, image, video and speech models.
How much usable memory does a Tesla V100 32GB actually have?
Its nameplate is 32 GB, but about 29.76 GiB is available to a model once driver and compositor overhead is accounted for.
Is a Tesla V100 32GB fast for local AI?
Its memory bandwidth is 900 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.