Sliding-window attention
Layers that attend only to a fixed recent window rather than the whole context.
Most layers in models like Gemma 3 attend over a fixed window — commonly 1024 tokens — while a minority attend over the full context. This dramatically reduces cache size at long context: a 27B model with 52 of 62 layers windowed needs about 3.1 GiB of cache at 32K, against 15.5 GiB if every layer cached everything.
Which layers are windowed is set by a period. That period comes from the model file when present and from a per-architecture default otherwise. A period of zero means every layer is windowed, not none — a detail that inverts the calculation if misread.