kimi-k3-in-c

kimi-k3-in-c runs a 2.78-trillion-parameter model on a machine with 8 GB of RAM, and the claim holds up. It's a from-scratch inference engine in portable C99, no BLAS, no GPU, no framework. Kimi K3 is a mixture-of-experts model where only a handful of experts fire per token, so the engine streams weights from disk and caches the few it needs. Apache 2.0, free.

The headline number is real but it's not the one that matters. You need roughly 1.7 TB of fast NVMe for the checkpoint, Linux on x86-64 with AVX2, and patience: generation runs seconds to tens of seconds per token depending on hardware and memory budget. More RAM buys real speed, but nothing close to interactive. Output is byte-identical at every memory budget, validated against a PyTorch reference, which is the kind of engineering discipline most projects skip.

Use this to learn how MoE inference actually works, to benchmark against, or to prove a point. Don't use it to serve anything: Moonshot's hosted API is faster and cheaper for real workloads, and llama.cpp is the right tool for models that fit your hardware.

The catch: it's a weeks-old, essentially solo reference implementation, and the model weights carry Moonshot's own license, not Apache. Treat it as a brilliant teaching artifact, not infrastructure.