coreai-models

Apple's coreai-models is the toolkit for getting AI models running on Apple hardware, on the device, not in the cloud. It gives you export recipes that convert popular open-source models into Apple's Core AI format, Python tools for authoring custom models in PyTorch, and a Swift package with the runtime pieces to actually run them on macOS and iOS. This is how you ship an app that does inference locally instead of calling out to an API.

There's nothing to host, it's a library and a set of recipes you build into your own app. The work is real but well-scoped: you convert a model, wire up the Swift runtime, and test it on-device. For developers already in Apple's ecosystem, this is the sanctioned path rather than fighting the conversion tooling yourself.

App developers who want on-device AI, for privacy, offline capability, or to skip per-call API costs, are exactly who this is for. If you're not shipping to Apple platforms, it's irrelevant, look at GGUF and llama.cpp or ONNX instead. The payoff here is models that run on the Neural Engine with no network round trip and no data leaving the phone.

The catch: this is Apple-only by design, so it ties you to their platforms and their format. On-device inference also means you're bounded by what the hardware can run, you're not loading a giant frontier model onto a phone. It's BSD-licensed and free, but it's a developer toolkit, there's nothing here unless you're building something.