MLA (multi-head latent attention)
An attention variant that caches a compressed latent instead of full keys and values.
Used by the DeepSeek family. Rather than caching per-head keys and values, MLA caches a low-rank latent projection, and allocates no value cache at all. Cache width is the latent rank plus the rotary dimensions, typically 576 elements per token per layer.
The trap is that these models still declare a large num_key_value_heads in their configuration. A calculator reading that field and multiplying will overstate the cache by more than an order of magnitude.