NVIDIA · datacenter

H100 SXM 80GB

H100 SXM 80GB has 80 GB of VRAM at 3350 GB/s — about 74.40 GiB usable after driver and compositor overhead. 2065 of 2118 indexed models fit at 128K context with q4_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
80 GB
HBM3
Bandwidth
3350 GB/s
5120-bit bus
Tensor FP16
989 TF
dense
TDP
700 W
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
vision language 186text 1775image 2audio asr 39audio tts 21video 16embedding 26

What fits at 128K context

largest quantization that fits, per model · 2065 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
GLM-4.6VMoEQ4_K_L108B66.89 GiB6.47 GiB74.38 GiB0.02 GiB73±37%
GLM-4.6-REAP-268B-A32BMoEUD-TQ1_0269B60.36 GiB12.94 GiB74.34 GiB0.06 GiB52±37%
WizardLM-Uncensored-SuperCOT-StoryTelling-30bQ4_K_M32.5B18.27 GiB54.84 GiB74.19 GiB0.21 GiB26±22%
Wizard-Vicuna-30B-UncensoredI1-Q4_K_M32.5B18.27 GiB54.84 GiB74.19 GiB0.21 GiB26±22%
archangel_sft-kto_llama30bI1-Q4_K_M32.5B18.27 GiB54.84 GiB74.19 GiB0.21 GiB26±22%
Qwen3.5-122B-A10BMoEQ4_K_M125B72.29 GiB0.84 GiB74.16 GiB0.24 GiB138±37%
MiniMax-M2.7MoEIQ2_S229B64.38 GiB8.72 GiB74.09 GiB0.31 GiB73±37%
grok-2MoEIQ2_XXS270B63.81 GiB9.00 GiB73.95 GiB0.45 GiB39±37%
Qwen3.5-REAP-262B-A17BMoEIQ2_XS262B71.81 GiB1.05 GiB73.92 GiB0.48 GiB137±37%
MiMo-V2-FlashMoEKV unresolvedIQ2_XXS310B68.47 GiB4.22 GiB73.74 GiB0.66 GiB100±37%
MiniMax-M2MoEUD-IQ1_M229B64.01 GiB8.72 GiB73.72 GiB0.68 GiB73±37%
Behemoth-X-123B-v2Q3_K_L123B60.12 GiB12.38 GiB73.65 GiB0.75 GiB26±22%
Mistral-Large-Instruct-2411Q3_K_L123B60.12 GiB12.38 GiB73.65 GiB0.75 GiB26±22%
command-a-plus-05-2026-bf16MoEIQ2_M219B71.32 GiB1.24 GiB73.57 GiB0.83 GiB106±37%
GLM-4.7-REAP-218B-A32BMoEIQ2_S218B59.49 GiB12.94 GiB73.47 GiB0.93 GiB51±37%
MiniMax-M2.1MoEUD-IQ1_M229B63.74 GiB8.72 GiB73.44 GiB0.96 GiB73±37%
MiniMax-M2.5MoEUD-IQ1_M229B63.74 GiB8.72 GiB73.44 GiB0.96 GiB73±37%
Step-3.5-Flash-REAP-121B-A11BI1-Q3_K_L121B58.48 GiB13.79 GiB73.30 GiB1.10 GiB26±22%
Qwen3.5-122B-A10B-hereticMoEI1-Q4_1123B71.35 GiB0.84 GiB73.22 GiB1.18 GiB140±37%
GLM-4.5VMoEI1-Q4_K_M108B65.61 GiB6.47 GiB73.11 GiB1.29 GiB74±37%
MiniMax-M2.7-BF16-ultra-uncensored-hereticMoEI1-IQ2_S229B63.36 GiB8.72 GiB73.07 GiB1.33 GiB73±37%
Qwen3-235B-A22B-abliteratedMoEI1-IQ2_S235B65.40 GiB6.61 GiB73.05 GiB1.35 GiB73±37%
command-r-35b-writer-v2I1-Q6_K35.0B26.74 GiB45.00 GiB72.84 GiB1.56 GiB27±22%
Step-3.7-FlashIQ2_S201B57.93 GiB13.79 GiB72.74 GiB1.66 GiB27±22%
Qwen3-VL-235B-A22B-ThinkingMoEUD-IQ1_M236B64.90 GiB6.61 GiB72.55 GiB1.85 GiB74±37%
OYM-Qimi-122B-A10B-K2.6MoEI1-Q4_K_M125B70.64 GiB0.84 GiB72.51 GiB1.89 GiB141±37%
Qwopus3.5-122B-A10B-Kimi-K2.6-destill-healed-abliteratedMoEQ4_K_M123B70.63 GiB0.84 GiB72.51 GiB1.89 GiB141±37%
Qwen3-VL-235B-A22B-InstructMoEUD-IQ1_M236B64.83 GiB6.61 GiB72.47 GiB1.93 GiB74±37%
Mistral-Medium-3.5-128BQ3_K_M128B58.94 GiB12.38 GiB72.47 GiB1.93 GiB27±22%
HuatuoGPT-o1-72BQ6_K72.7B59.93 GiB11.25 GiB72.31 GiB2.09 GiB27±22%
Rombo-LLM-V3.0-Qwen-72bQ6_K72.7B59.93 GiB11.25 GiB72.31 GiB2.09 GiB27±22%
Qwen2.5-72B-Instruct-abliteratedQ6_K72.7B59.93 GiB11.25 GiB72.31 GiB2.09 GiB27±22%
EVA-Qwen2.5-72B-v0.2Q6_K72.7B59.93 GiB11.25 GiB72.31 GiB2.09 GiB27±22%
MiroThinker-v1.0-72BQ6_K72.7B59.93 GiB11.25 GiB72.31 GiB2.09 GiB27±22%
Qwen2.5-Math-72B-InstructQ6_K72.7B59.93 GiB11.25 GiB72.31 GiB2.09 GiB27±22%
Qwen2.5-72B-InstructQ6_K72.7B59.93 GiB11.25 GiB72.31 GiB2.09 GiB27±22%
Qwen2.5-72BQ6_K72.7B59.93 GiB11.25 GiB72.31 GiB2.09 GiB27±22%
Kimi-Dev-72BQ6_K72.7B59.93 GiB11.25 GiB72.31 GiB2.09 GiB27±22%
magnum-v4-72bQ6_K72.7B59.93 GiB11.25 GiB72.31 GiB2.09 GiB27±22%
KAT-Dev-72B-ExpQ6_K72.7B59.93 GiB11.25 GiB72.31 GiB2.09 GiB27±22%
Chuluun-Qwen2.5-72B-v0.01Q6_K72.7B59.93 GiB11.25 GiB72.31 GiB2.09 GiB27±22%
Homer-v1.0-Qwen2.5-72BQ6_K72.7B59.93 GiB11.25 GiB72.31 GiB2.09 GiB27±22%
Qwen2.5-VL-72B-InstructQ6_K73.4B59.93 GiB11.25 GiB72.31 GiB2.09 GiB27±22%
Tower-Plus-72B-ultra-uncensored-hereticI1-Q6_K72.7B59.93 GiB11.25 GiB72.31 GiB2.09 GiB27±22%
Chronos-Platinum-72BQ6_K72.7B59.93 GiB11.25 GiB72.31 GiB2.09 GiB27±22%
UI-TARS-72B-DPOQ6_K73.4B59.93 GiB11.25 GiB72.31 GiB2.09 GiB27±22%
MiMo-V2.5MoEKV unresolvedIQ1_M311B67.01 GiB4.22 GiB72.28 GiB2.12 GiB101±37%
GLM-4.5-Air-DerestrictedMoEQ4_1110B64.77 GiB6.47 GiB72.27 GiB2.13 GiB74±37%
GLM-4.5-AirMoEQ4_1110B64.77 GiB6.47 GiB72.27 GiB2.13 GiB74±37%
Llama-4-Scout-17B-16E-InstructMoEKV unresolvedQ4_1109B64.35 GiB6.75 GiB72.13 GiB2.27 GiB73±37%
Mixtral-8x22B-Instruct-v0.1MoEQ3_K_M141B63.14 GiB7.88 GiB72.07 GiB2.33 GiB41±37%
Mixtral-8x22B-v0.1MoEQ3_K_M141B63.14 GiB7.88 GiB72.07 GiB2.33 GiB41±37%
Mixtral-8x22B-v0.1MoEQ3_K_M141B63.13 GiB7.88 GiB72.07 GiB2.33 GiB41±37%
Hy3MoEIQ1_S299B59.65 GiB11.25 GiB71.94 GiB2.46 GiB62±37%
MiniMax-M2.1-REAP-139B-A10BMoEI1-Q3_K_M139B62.01 GiB8.72 GiB71.72 GiB2.68 GiB68±37%
m51Lab-MiniMax-M2.7-REAP-139B-A10BMoEI1-Q3_K_M139B62.01 GiB8.72 GiB71.72 GiB2.68 GiB68±37%
Laguna-S-2.1MoEQ4_1118B68.96 GiB1.73 GiB71.71 GiB2.69 GiB122±37%
GLM-Z1-Rumination-32B-0414BF1633.1B61.74 GiB8.58 GiB71.41 GiB2.99 GiB27±22%
Ornith-1.0-35B-AEON-Ultimate-Uncensored-BF16MoEQ8_035.1B69.57 GiB0.70 GiB71.28 GiB3.12 GiB145±37%
CallerBF1632.8B61.04 GiB9.00 GiB71.13 GiB3.27 GiB27±22%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a H100 SXM 80GB run?
2065 of 2118 indexed open-weight models fit a H100 SXM 80GB at 131,072 context with q4_0 KV cache, the largest being GLM-4.6V at Q4_K_L. That covers text, vision-language, image, video and speech models.
How much usable memory does a H100 SXM 80GB actually have?
Its nameplate is 80 GB, but about 74.40 GiB is available to a model once driver and compositor overhead is accounted for.
Is a H100 SXM 80GB fast for local AI?
Its memory bandwidth is 3350 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.