Is GPT-NeoX-20B-Erebus or Qwen3-Coder-30B-A3B-Instruct smaller?

Both publish quantizations; compare the table for the exact sizes.

Which uses less memory at long context?

Qwen3-Coder-30B-A3B-Instruct — its KV cache at 32K context is 11.0x smaller, because of how its attention layers cache.

Model comparison

GPT-NeoX-20B-Erebus vs Qwen3-Coder-30B-A3B-Instruct

These two publish different quantization sets; the table below has the exact sizes. At long context the gap widens: Qwen3-Coder-30B-A3B-Instruct's KV cache at 32K is 11.0× smaller, which usually matters more than the difference in weights.

From the file· summed bytes, KV per layer

Side by side

	GPT-NeoX-20B-Erebus	Qwen3-Coder-30B-A3B-Instruct
Parameters	20.6B	30.5B
Architecture	gptneox	qwen3moe
Layers	44	48
Native context	2,048	262,144
Mixture of experts	no	yes, 128 experts
Quantizations published	23	46
Smallest quantization	4.12 GiB	7.46 GiB
Q4_K_M	—	17.28 GiB
Licence	apache-2.0	apache-2.0

KV cache by context

the term that decides long-context viability

Context	GPT-NeoX-20B-Erebus	Qwen3-Coder-30B-A3B-Instruct	Ratio
4,096	4.13 GiB	0.38 GiB	11.00×
8,192	8.25 GiB	0.75 GiB	11.00×
16,384	16.50 GiB	1.50 GiB	11.00×
32,768	33.00 GiB	3.00 GiB	11.00×
65,536	66.00 GiB	6.00 GiB	11.00×
131,072	132.00 GiB	12.00 GiB	11.00×