HRM-Text
HRM-Text is a 1B text generation model based on the HRM architecture, strengthened by task completion and latent space reasoning.
The Lens
HRM-Text is a pretraining framework for building your own small language model from scratch, cheaply. The claim is foundation-model pretraining with 130 to 600 times less compute and far less data than usual, using a Hierarchical Reasoning Model architecture instead of a standard transformer. Apache-2.0 and free, with a pre-trained 1B checkpoint on Hugging Face if you'd rather not train.
This is research infrastructure, not a product. It's a full training pipeline: prepare tokenized data, launch distributed training, evaluate on benchmarks like MATH and MMLU, export to Hugging Face format. The architecture leans on recurrent reasoning layers, sequence packing, and FlashAttention 3 kernels to squeeze efficiency out of the run.
The catch: cheap is relative. You still need a cluster of 8 to 16 H100 GPUs and roughly 800 to 1,500 dollars in compute for a full training run. This is for researchers and teams exploring efficient architectures, not for anyone who just wants to use a model. If that's you, grab the checkpoint and skip the training code.
Free vs Self-Hosted vs Paid
fully freeFree: Apache-2.0, free. The full pretraining framework, plus a pre-trained 1B checkpoint on Hugging Face.
Self-hosted: This is the only way, and it's heavy. A training run needs a cluster of 8 to 16 H100 GPUs and the infrastructure to run distributed training.
Paid: No paid tier. Your cost is GPU compute, roughly $800 to $1,500 per full run at typical H100 rates.
The framework is free. A full training run still costs roughly $800-$1,500 in GPU time.
Get tools like this every Wednesday
One featured tool, three on the radar. No fluff.
License: Apache License 2.0
Use freely. Patent grant included.
Commercial use: ✓ Yes
About
- Owner
- sapientinc (Organization)
- Stars
- 1,636
- Forks
- 156