LIMO: Less is More for Reasoning
817 curated samples lift a base model to 63.3% on AIME24 and 95.6% on MATH500, arguing pretraining already holds the reasoning knowledge.
Proposes the 'Less-Is-More Reasoning Hypothesis', which holds that when pretraining is rich in math, a few cognitively well-designed demonstrations suffice to elicit long reasoning. Fed the 'RL elicits, doesn't create' debate alongside s1.
- Date
- Wednesday, 5 February 2025
- Lab
- SJTU GAIR
- Kind
- paper
- Access
- open weights
Figures
| Measure | Value | Measured by |
|---|---|---|
| AIME24 | 63.3% vs 6.5% from prior SFT recipes using 100x more data | authors |
| MATH500 | 95.6% vs 59.2% prior | authors |
COLM 2025. Lead author Yixin Ye, senior author Pengfei Liu. arXiv v1 2025-02-05.
Sources
This record was checked against its sources on 6 October 2026. How we check