vLLM

vLLM is the production LLM inference engine that made PagedAttention mainstream. If you're serving models to multiple users simultaneously, vLLM's memory-efficient KV cache management means you can serve 2-4x more concurrent requests on the same GPU than naive implementations. 74k stars and adopted as the default by Hugging Face.

TGI (Hugging Face's own server) has a faster Rust core with 1-5ms less overhead per request — better for low-latency single-user serving. TensorRT-LLM squeezes 20-40% more throughput on NVIDIA hardware but requires hour-long compilation steps. Ollama is simpler for local dev but not built for production scale.

Use vLLM if you're self-hosting LLMs for a product with multiple concurrent users. The OpenAI-compatible API means your app code doesn't change.

The catch: it's Python-based, so scheduling adds 1-5ms latency per request versus TGI's Rust core. GPU required — this isn't for CPU-only setups. And model support, while broad, doesn't always include the latest architecture on day one. SGLang is emerging as a faster alternative for some workloads.