FlagGems

FlagGems is a drop-in speed layer for PyTorch. It is a collection of hand-tuned math operators, the low-level kernels that do the heavy lifting inside a neural net, written in Triton instead of raw CUDA. Register it with PyTorch's ATen backend and your existing model code runs on top of these kernels with no API changes. All Apache-2.0, all free.

The pitch beyond raw speed is portability. Most kernel libraries are locked to NVIDIA. FlagGems is backend-neutral and claims over ten hardware backends, so the same operators can target non-NVIDIA accelerators. That matters if you are trying to get off the NVIDIA-only path or run on domestic silicon. It is eager-mode ready, so you do not need torch.compile to get the benefit. The cost is setup: you need Triton and a compatible toolchain, and both coverage and performance vary by operator and by backend.

Who should use it: if you train or serve LLMs and want a portability hedge or a free speed bump on non-CUDA hardware, it is worth testing. Solo on a single NVIDIA card, the existing kernels are probably fine and the upside is small. Teams running multi-vendor hardware, or planning around GPU supply, are the real audience.

The catch: FlagGems comes from FlagOS, backed by BAAI, a Beijing AI research institute. Operator coverage is broad but the hand-optimization is selective, so your actual speedup depends entirely on which operators your model leans on. Benchmark your own workload before you bet a training run on it.