18 August 2026
Study finds high-quality data repetition scales with model size
- Researchers discovered that the best amount of times to repeat high-quality data grows slightly as models get larger, when keeping the same token-per-parameter ratio.
- Smaller test models can predict optimal repetition schedules for much larger models, potentially saving computation time and cost.
