UniRL

UniRL is a training framework for teaching AI models new behavior through reinforcement learning, across every major model type at once. Most RL training code is built for one kind of model. UniRL, from Tencent's Hunyuan team, runs a single RL loop over image and video diffusion models, large language models, vision-language models, and unified multimodal models. It ships real algorithms (Flow-DPPO, DRPO, CPPO) each backed by a paper, alongside standard baselines like GRPO.

There is nothing hosted here, and the ops burden is heavy. This is distributed training infrastructure: Ray for orchestration, FSDP for sharding models across GPUs, Hydra for configuration. You need a real GPU cluster and the experience to drive it. Getting a run going is a project, not an afternoon.

ML engineers and researchers doing RL post-training on multimodal models are the only audience, and for them the value is having one framework instead of stitching together separate pipelines per model family. If you're not training models at this level, this is not for you. There is no lighter on-ramp; the complexity is inherent to the work.

The catch: it's research-grade infrastructure, Apache-licensed and free, but free in the way a jet engine is free if someone hands you one. The cost is the cluster, the GPU hours, and the expertise. Expect rough edges, sparse docs, and the assumption that you already know how RL training works.