MoonEP

MoonEP is a low-level building block for running Mixture-of-Experts AI models across many GPUs. Its whole job is keeping the token load balanced across GPU ranks, using dynamic redundant experts and fused kernels, so no single GPU becomes the bottleneck during MoE inference or training. It comes from Moonshot AI, MIT-licensed and free.

This is infrastructure, not an app. There's nothing to click. You need a datacenter-grade setup: a multi-GPU NVIDIA cluster with NVLink, since it relies on symmetric memory and remote reads between GPUs. It was tested on eight H20s. If that sentence didn't apply to you, this isn't your tool.

For the specific teams training or serving large MoE models on GPU clusters, MoonEP is a serious performance primitive that competes with libraries like DeepEP. Everyone else can skip it. There's no team-size question here, it's for infrastructure engineers at labs and companies running big models.

The catch is reach and maturity. It's NVIDIA-only today, with other hardware listed as coming soon, and it's very new, benchmarked against DeepEP but not yet widely battle-tested. It's a sharp tool for a narrow, expensive job, not something you casually adopt.