Ollama
llama.cpp wrapped in a daemon and a model registry. Convenient, at the cost of some control and some lag.
From the file· capabilities, not benchmarks
What it is
Runs as a background service with a simple pull-and-run workflow and an OpenAI-compatible endpoint. It is the easiest way to get a model running and the most common entry point into local AI.
Capabilities
| Model formats | GGUF |
| KV cache quantization | yes |
| CPU offload | yes |
| MoE expert offload | not supported |
| Multi-GPU | Inherits llama.cpp's layer split; little direct control. |
| Concurrency | Fine for a handful of concurrent requests, not for a production workload. |
| Platforms | Linux, macOS, Windows |
Best for
Getting started, and any application that wants a local OpenAI-compatible endpoint without managing the engine.
Watch out for
Its short model names map to specific quantizations that are not obvious — a bare tag is usually a 4-bit build, not the model's best available. It also lags upstream, so a very new architecture may not load yet even though llama.cpp supports it.