18 August 2026
Repeating quality training data scales slightly with model size
- Researchers found that the best amount of times to repeat high-quality data during training increases modestly as models grow larger, when keeping the total training volume constant.
- Smaller test models can predict the optimal repetition strategy for much larger models, potentially saving computation time and resources during development.
