Does Qwen2.5-72B-Instruct-abliterated fit in 24GB of VRAM?

Figures are GiB of total memory: weights plus KV cache plus compute buffer and backend overhead. Weights and KV are near-exact; the overhead term is modeled. Hover any cell for the breakdown.

A capacity is not a card. Whether a model fits depends only on memory, so every figure above holds for any 24GB accelerator. How fast it runs depends on memory bandwidth, which varies several-fold between cards of the same capacity — so putting a tokens-per-second number here would be inventing one. Pick a specific card from hardware and the speed column appears.

A parameters × bits ÷ 8 estimate ignores two things that dominate at long context. First, the weights themselves are not the nominal rate — quantizations are mixtures, so the real file is consistently larger than the label implies. Second, most of this model's layers cache only a 131,072-token window rather than the full context.