llama.cpp

architecture behaviour and quantization type table

Byte counts are exact, summed across all shards of a quantization.

KV inputs are architectural facts; the memory total for a given context is computed, not stored.

No measured performance data is included — this release contains no measured figures.


conv conv full_attention conv conv conv full_attention conv conv conv full_attention conv conv conv full_attention conv conv conv full_attention conv conv full_attention conv conv


mamba mamba mamba mamba mamba attention mamba mamba mamba mamba mamba mamba mamba mamba mamba attention mamba mamba mamba mamba mamba mamba mamba mamba mamba attention mamba mamba mamba mamba mamba mamba mamba mamba mamba attention mamba mamba mamba mamba


attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention


conv conv full_attention conv conv full_attention conv conv conv full_attention conv conv conv full_attention conv conv conv full_attention conv conv conv full_attention conv conv full_attention conv conv full_attention conv conv


attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention attention


conv conv full_attention conv conv conv full_attention conv conv conv full_attention conv conv conv full_attention conv conv conv full_attention conv conv conv full_attention conv conv conv full_attention conv conv conv full_attention conv conv conv full_attention conv conv conv full_attention conv


hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid hybrid


mamba mamba mamba mamba mamba mamba mamba mamba mamba mamba attention mamba mamba attention mamba mamba mamba attention mamba mamba mamba mamba mamba mamba mamba mamba mamba attention mamba mamba mamba mamba


RTX PRO 6000 Blackwell Workstation Edition


RTX PRO 6000 Blackwell Max-Q Workstation Edition