omlx
LLM inference server with continuous batching and SSD caching for Apple Silicon, managed from the macOS menu bar.
The Lens
By Erik Loyd, SaaS CEO and former COO/CFO of an AWS Premier Partner.
Updated Oct 2026
oMLX runs large language models on your Mac and manages them from the menu bar. You get a local API that speaks both the OpenAI and Anthropic formats, so Claude Code, Codex or Cursor can point at your own machine instead of a paid cloud model. Apache 2.0, no account, no paid tier.
The reason to pick it over Ollama is the cache. Coding agents keep resending the same long prompt, and oMLX saves that work to your SSD, so a returning context loads from disk instead of being recomputed, even after a restart. The signed DMG auto-updates and ships the fast native kernels prebuilt.
Solo developers running a local coding assistant get the most from it. Small teams can share one big Mac over the API; it refuses network binds without an API key. Larger teams belong on vLLM and real GPUs.
The catch: Apple Silicon, macOS 15 or newer, and the project calls 16GB of RAM the minimum with 64GB recommended. Homebrew and source installs without full Xcode skip the native kernels, and GLM-5.2 prefill then runs roughly 30x slower. Take the DMG. It's also one person's after-hours project.
Free vs Self-Hosted vs Paid
fully freeFree: Everything. Apache 2.0, no account, no license key, no paid tier. The DMG, the Homebrew formula and a source build are the same software; the DMG just ships the native Metal kernels precompiled.
Self-hosted: Your Mac. Apple Silicon (M1 through M5), macOS 15.0 or newer, and Python 3.11 to 3.13 for source installs. The project's own FAQ puts the floor at 16GB of RAM and recommends 64GB or more for larger models. Weights download free from Hugging Face. Building the native kernels needs full Xcode with the Metal toolchain; Command Line Tools alone won't do it. Watch your disk: the SSD cache defaults to auto, which can use up to half of your free disk space.
Paid: There is no paid tier and no donation link in the README. Spend money when you outgrow one machine: a hosted model API or rented GPUs beats buying a second Mac for occasional heavy jobs.
Free under Apache 2.0 with no paid tier. The real cost is the Mac: Apple Silicon, macOS 15 or newer, and the project recommends 64GB of RAM for comfortable use.
What to do by team size
- Solo
- free; the DMG installs in minutes and ships the fast kernels prebuilt
- Small team
- free; one high-memory Mac serves the team over the API, and network binds require an API key
- Medium team
- free, but put it behind a reverse proxy and expect a single Mac to bottleneck under real concurrency
- Large team
- free to try, wrong thing to standardize on; multi-Mac clustering is an experimental source-build preview. Use vLLM on GPUs
Get tools like this every Wednesday
One featured tool, three on the radar. No fluff.
Similar Tools
A low score is not a verdict on quality. Young and niche tools start low by design. How we calculate scores
Trust Signals
About
- Stars
- 23,404
- Forks
- 2,101
Explore Further
More tools in the directory
rea
Reverse engineer anything with agents, from app behavior down to native binaries.
435.8k ★openclaw
Your own personal AI assistant. Any OS. Any Platform. The lobster way. 🦞
394.0k ★everything-claude-code
The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond.
298.1k ★Featured in The Open Source Drop #22