Model comparison

gemma-4-12B-it-qat-q4_0-unquantized-assistant vs Llama-3.2-1B-Instruct

These two publish different quantization sets; the table below has the exact sizes. At long context the gap widens: gemma-4-12B-it-qat-q4_0-unquantized-assistant's KV cache at 32K is 3.5× smaller, which usually matters more than the difference in weights.

From the file· summed bytes, KV per layer

Side by side

gemma-4-12B-it-qat-q4_0-unquantized-assistantLlama-3.2-1B-Instruct
Parameters423M1.2B
Architecturegemma4-assistantllama
Layers416
Native context262,144131,072
Mixture of expertsnono
Quantizations published339
Smallest quantization0.30 GiB0.39 GiB
Q4_K_M—0.75 GiB
Licenceapache-2.0—

KV cache by context

the term that decides long-context viability
Contextgemma-4-12B-it-qat-q4_0-unquantized-assistantLlama-3.2-1B-InstructRatio
4,0960.07 GiB0.13 GiB1.88×
8,1920.10 GiB0.25 GiB2.56×
16,3840.16 GiB0.50 GiB3.12×
32,7680.29 GiB1.00 GiB3.51×
65,5360.54 GiB2.00 GiB3.74×
131,0721.04 GiB4.00 GiB3.86×
gemma-4-12B-it-qat-q4_0-unquantized-assistant vs Llama-3.2-1B-Instruct — size, memory and hardware fit — ossmodeldb