NVIDIA · datacenter

H100 SXM 80GB

H100 SXM 80GB has 80 GB of VRAM at 3350 GB/s — about 74.40 GiB usable after driver and compositor overhead. 2053 of 2118 indexed models fit at 128K context with q8_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
80 GB
HBM3
Bandwidth
3350 GB/s
5120-bit bus
Tensor FP16
989 TF
dense
TDP
700 W
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 1763vision language 186image 2audio asr 39audio tts 21video 16embedding 26

What fits at 128K context

largest quantization that fits, per model · 2053 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
MiniMax-M2.1-REAP-139B-A10BMoEI1-IQ3_M139B56.81 GiB16.47 GiB74.27 GiB0.13 GiB47±37%
m51Lab-MiniMax-M2.7-REAP-139B-A10BMoEI1-IQ3_M139B56.81 GiB16.47 GiB74.27 GiB0.13 GiB47±37%
v6-Finch-14B-HFQ4_K_L14.1B8.35 GiB64.81 GiB74.21 GiB0.19 GiB26±22%
MiniMax-M2.7MoEIQ2_XXS229B56.67 GiB16.47 GiB74.12 GiB0.28 GiB50±37%
Qwen3.5-122B-A10B-hereticMoEI1-Q4_1123B71.35 GiB1.59 GiB73.97 GiB0.43 GiB128±37%
Skyfall-31B-v4.2BF1631.4B58.41 GiB14.34 GiB73.87 GiB0.53 GiB26±22%
Behemoth-X-123B-v2Q3_K_S123B49.22 GiB23.38 GiB73.75 GiB0.65 GiB26±22%
Mistral-Large-Instruct-2411Q3_K_S123B49.22 GiB23.38 GiB73.75 GiB0.65 GiB26±22%
c4ai-command-r-plus-08-2024Q4_K_S104B55.55 GiB17.00 GiB73.73 GiB0.67 GiB26±22%
MiMo-V2-FlashMoEKV unresolvedI1-IQ1_M310B64.64 GiB7.97 GiB73.66 GiB0.74 GiB77±37%
GLM-4.5VMoEI1-Q4_K_S108B60.29 GiB12.22 GiB73.54 GiB0.86 GiB55±37%
MiniMax-M2.7-BF16-ultra-uncensored-hereticMoEI1-IQ2_XXS229B55.99 GiB16.47 GiB73.44 GiB0.96 GiB50±37%
MiniMax-M2.1MoEI1-IQ2_XXS229B55.99 GiB16.47 GiB73.44 GiB0.96 GiB50±37%
MiniMax-M2.5MoEI1-IQ2_XXS229B55.99 GiB16.47 GiB73.44 GiB0.96 GiB50±37%
Qwen2.5-72BQ5_172.7B50.88 GiB21.25 GiB73.26 GiB1.14 GiB26±22%
OYM-Qimi-122B-A10B-K2.6MoEI1-Q4_K_M125B70.64 GiB1.59 GiB73.26 GiB1.14 GiB129±37%
Qwopus3.5-122B-A10B-Kimi-K2.6-destill-healed-abliteratedMoEQ4_K_M123B70.63 GiB1.59 GiB73.26 GiB1.14 GiB129±37%
Laguna-S-2.1MoEQ4_1118B68.96 GiB3.26 GiB73.24 GiB1.16 GiB104±37%
Mixtral-8x22B-Instruct-v0.1MoEQ3_K_S141B57.28 GiB14.88 GiB73.22 GiB1.18 GiB34±37%
Mixtral-8x22B-v0.1MoEQ3_K_S141B57.28 GiB14.88 GiB73.22 GiB1.18 GiB34±37%
GLM-4.5-Air-REAP-82B-A12BMoEQ5_K_M81.9B59.97 GiB12.22 GiB73.21 GiB1.19 GiB52±37%
Mixtral-8x22B-v0.1MoEQ3_K_S141B57.27 GiB14.88 GiB73.21 GiB1.19 GiB34±37%
Mistral-Medium-3.5-128BIQ3_XXS128B48.59 GiB23.38 GiB73.12 GiB1.28 GiB26±22%
HuatuoGPT-o1-72BQ5_K_M72.7B50.71 GiB21.25 GiB73.09 GiB1.31 GiB26±22%
Rombo-LLM-V3.0-Qwen-72bQ5_K_M72.7B50.71 GiB21.25 GiB73.09 GiB1.31 GiB26±22%
EVA-Qwen2.5-72B-v0.2Q5_K_M72.7B50.71 GiB21.25 GiB73.09 GiB1.31 GiB26±22%
Qwen2.5-72B-Instruct-abliteratedQ5_K_M72.7B50.71 GiB21.25 GiB73.09 GiB1.31 GiB26±22%
MiroThinker-v1.0-72BQ5_K_M72.7B50.71 GiB21.25 GiB73.09 GiB1.31 GiB26±22%
Qwen2.5-Math-72B-InstructQ5_K_M72.7B50.71 GiB21.25 GiB73.09 GiB1.31 GiB26±22%
Qwen2.5-72B-InstructQ5_K_M72.7B50.71 GiB21.25 GiB73.09 GiB1.31 GiB26±22%
magnum-v4-72bQ5_K_M72.7B50.71 GiB21.25 GiB73.09 GiB1.31 GiB26±22%
KAT-Dev-72B-ExpQ5_K_M72.7B50.71 GiB21.25 GiB73.09 GiB1.31 GiB26±22%
Homer-v1.0-Qwen2.5-72BQ5_K_M72.7B50.71 GiB21.25 GiB73.09 GiB1.31 GiB26±22%
Chuluun-Qwen2.5-72B-v0.01Q5_K_M72.7B50.71 GiB21.25 GiB73.09 GiB1.31 GiB26±22%
Qwen2.5-VL-72B-InstructQ5_K_M73.4B50.71 GiB21.25 GiB73.09 GiB1.31 GiB26±22%
Tower-Plus-72B-ultra-uncensored-hereticI1-Q5_K_M72.7B50.71 GiB21.25 GiB73.09 GiB1.31 GiB26±22%
Chronos-Platinum-72BQ5_K_M72.7B50.71 GiB21.25 GiB73.09 GiB1.31 GiB26±22%
UI-TARS-72B-DPOQ5_K_M73.4B50.71 GiB21.25 GiB73.09 GiB1.31 GiB26±22%
Step-3.5-Flash-REAP-121B-A11BI1-IQ3_XS121B45.92 GiB26.05 GiB73.00 GiB1.40 GiB26±22%
NVIDIA-Nemotron-3-Super-120B-A12B-BF16MoEQ4_0124B66.12 GiB5.84 GiB72.96 GiB1.44 GiB84±37%
Devstral-2-123B-Instruct-2512IQ3_XS125B48.11 GiB23.38 GiB72.64 GiB1.76 GiB27±22%
XORTRON-NXTXPRTXXLI1-IQ3_XS128B48.11 GiB23.38 GiB72.64 GiB1.76 GiB27±22%
GLM-4.5-Air-DerestrictedMoEQ4_0110B59.38 GiB12.22 GiB72.63 GiB1.77 GiB55±37%
GLM-4.5-AirMoEQ4_0110B59.38 GiB12.22 GiB72.63 GiB1.77 GiB55±37%
Llama-4-Scout-17B-16E-InstructMoEKV unresolvedQ4_0109B58.72 GiB12.75 GiB72.50 GiB1.90 GiB54±37%
Qwen3.5-122B-A10BMoEQ4_K_S125B69.66 GiB1.59 GiB72.29 GiB2.11 GiB130±37%
Qwen3-VL-235B-A22B-ThinkingMoEUD-IQ1_S236B58.65 GiB12.48 GiB72.17 GiB2.23 GiB55±37%
Gemma-4-Novelist-Eclipse-31BBF1632.7B59.82 GiB11.25 GiB72.15 GiB2.25 GiB27±22%
Gemma-4-31B-StyleTuneBF1632.7B59.82 GiB11.25 GiB72.15 GiB2.25 GiB27±22%
calme-2.3-rys-78bQ4_K_L78.0B48.08 GiB22.84 GiB72.05 GiB2.35 GiB27±22%
GLM-4.7-REAP-218B-A32BMoEIQ1_M218B46.56 GiB24.44 GiB72.04 GiB2.36 GiB35±37%
Qwen3-VL-235B-A22B-InstructMoEUD-IQ1_S236B58.49 GiB12.48 GiB72.01 GiB2.39 GiB55±37%
Ornith-1.0-35B-AEON-Ultimate-Uncensored-BF16MoEQ8_035.1B69.57 GiB1.33 GiB71.91 GiB2.49 GiB134±37%
c4ai-command-r-08-2024F1632.3B60.17 GiB10.63 GiB71.91 GiB2.49 GiB27±22%
granite-4.1-30bBF1628.9B53.77 GiB17.00 GiB71.88 GiB2.52 GiB27±22%
gpt-oss-120b-Uncensored-xCloudMoEI1-Q4_1117B68.42 GiB2.40 GiB71.81 GiB2.59 GiB120±37%
gpt-oss-120b-abliteratedMoEI1-Q4_1117B68.42 GiB2.40 GiB71.81 GiB2.59 GiB120±37%
step-3.5-flashIQ2_XXS199B44.74 GiB26.05 GiB71.81 GiB2.59 GiB27±22%
Meta-Llama-3-70B-InstructQ5_170.6B49.37 GiB21.25 GiB71.74 GiB2.66 GiB27±22%
Llama-3.1-70BQ5_170.6B49.36 GiB21.25 GiB71.74 GiB2.66 GiB27±22%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a H100 SXM 80GB run?
2053 of 2118 indexed open-weight models fit a H100 SXM 80GB at 131,072 context with q8_0 KV cache, the largest being MiniMax-M2.1-REAP-139B-A10B at I1-IQ3_M. That covers text, vision-language, image, video and speech models.
How much usable memory does a H100 SXM 80GB actually have?
Its nameplate is 80 GB, but about 74.40 GiB is available to a model once driver and compositor overhead is accounted for.
Is a H100 SXM 80GB fast for local AI?
Its memory bandwidth is 3350 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.