CPU offload
Keeping some layers in system RAM when a model does not fit entirely in VRAM.
When a model exceeds available VRAM, layers can be held in system memory and executed on the CPU. The common belief that PCIe bandwidth is the bottleneck is wrong: weights are not streamed across the bus per token, and only a small activation vector crosses at each layer boundary. The real cost is that CPU memory bandwidth is far lower than GPU memory bandwidth, so the offloaded portion runs at its own much slower rate.
The result behaves like two rooflines in series, which is why offloading a little is cheap and offloading a lot is not.