deltafin

Deltafin runs Moonshot's Kimi K3, a massive trillion-parameter model, locally on a single machine and puts an OpenAI-compatible API in front of it. That means your local chat or coding agent can talk to a frontier-scale model running entirely on your own hardware, no API keys, no data leaving the box. It's MIT-licensed and free.

The honesty starts with the requirements. You need at least 64GB of RAM and about 1.7TB of storage for the full model, or roughly 215GB in streaming mode. It'll use Metal or CUDA if you have it and fall back to CPU if you don't. This is a heavy lift, firmly in enthusiast-with-a-workstation territory.

For a hobbyist who wants to run a giant model fully private and doesn't mind the setup, it's a fascinating way to do it for free instead of paying for hosted API calls. It's a proof of concept, not a team server, so treat it as an experiment rather than infrastructure.

The catch is speed and maturity, and the README is refreshingly blunt about it: set your client timeouts to hours, not seconds. It does greedy decoding only, handles one request at a time, and re-processes the full context every turn. It's an impressive hack, not a practical replacement for a hosted API when you need answers now.