waste

waste does something that shouldn't be possible on a laptop: run the full 2.8-trillion-parameter Kimi K3 model on modest RAM by streaming the activated expert weights straight off an NVMe drive. It's a dependency-free, embeddable C inference engine with documented multimodal (vision) support, from the team behind sqlite.ai. The headline isn't hype, Kimi K3 is a real Moonshot model released in mid-2026.

The trick is that a mixture-of-experts model only lights up a fraction of its weights per token, so waste keeps the cold weights on disk and streams what it needs. That's how a 3-trillion-class model fits on consumer hardware.

Free under Apache 2.0, for anyone at any scale who wants to poke at frontier-size models without a datacenter.

The catch is speed, and it's a big one. You're looking at roughly half a token per second on a 64GB MacBook, and you need close to a terabyte of NVMe to hold the model. It's explicitly pre-production, the format and API aren't frozen. This is a fascinating proof that disk-streaming inference works, not something you'd build a product on. Run it to learn, not to serve.