Rapid-MLX

Rapid-MLX runs open models locally on Apple Silicon, and speed is the whole pitch. One command installs it (Homebrew, pip, or a curl script), one command serves an OpenAI-compatible API, and it also speaks the Anthropic messages format, so Claude Code, Codex CLI, and Aider can point at it as a drop-in backend. Apache 2.0, completely free.

The engineering focus is on the things that make local agent workflows fail in practice. A prompt cache brings claimed time-to-first-token under a tenth of a second on repeat prompts, and tool calling is handled across 17 parser formats, which matters because local models break agent loops on malformed tool calls far more often than on reasoning. It's in Homebrew core and publishes to PyPI, with releases landing near daily.

Mac-only by design. If you run agents against local models on an M-series machine, try it head-to-head with Ollama or LM Studio; the performance claims ship with benchmarks in the repo rather than vibes. On Linux or Windows, look at Ollama or vLLM instead.

The catch: it's young and moves extremely fast, the headline numbers are the project's own, and independent benchmarks are still thin. And no inference engine fixes the ceiling: a fast small model is still a small model.