miles

Miles is a reinforcement-learning framework for post-training large language and vision models, the tuning step that happens after a base model is trained. It pairs SGLang for fast rollout with Megatron-LM for scalable training, and it is built for the trillion-parameter end of the spectrum, with day-0 support for models like DeepSeek-V4 and Kimi K3. Apache-2.0, and it co-evolves with the slime project it forked from.

This is heavy infrastructure. Fully async RL, MXFP8 and NVFP4 low-precision training, P2P RDMA weight transfer that pushes new weights to inference engines in seconds, LoRA and multi-LoRA. None of that is plug-and-play. You need a real GPU cluster, deep familiarity with Megatron and distributed training, and the patience to run something at frontier scale. A PyTorch FSDP2 backend exists for smaller HuggingFace-style runs, but the recipes and the big models live on Megatron.

If you are an ML team actually doing RL post-training on large models, Miles is aimed squarely at you and moves fast on new architectures. Everyone else, this is not the tool, and it is not pretending to be. The catch: "enterprise-grade" here means enterprise-scale hardware and expertise. On a couple of GPUs you will barely touch what it is for.