marin

Marin is an open recipe for training foundation models from scratch. Not just the finished weights, the whole pipeline: data curation, filtering, tokenization, pretraining, posttraining, and evaluation. It is Apache-2.0, and the thing that sets it apart is radical openness. Every run's checkpoints, data mixtures, and decisions get published, failed experiments included.

This is not a weekend project. It is built for serious compute, trained on Google's TPU Research Cloud, with the current focus a mixture-of-experts model north of 500 billion parameters. Realistically you use it two ways: as a library to run your own experiments (people have already forked it for DNA and protein models), or as a reference to learn how frontier models actually get built. Running the full recipe means real cluster time and a real compute bill.

Who it is for: researchers and teams who want a documented, reproducible path to training models, not an API to call. A solo builder who just wants to use a model should skip this and reach for an inference tool. Labs, grad students, and companies doing pretraining research get one of the most transparent open frameworks going, and the write-ups alone are worth the read.

The catch: the software is free, the compute is not. Reproducing even the smaller runs assumes access to TPUs or a comparable cluster. Marin's value is the openness and the methodology, not a shortcut around the hardware bill.