17 August 2026
New benchmark tests AI models on learning hidden rules through exploration
- Researchers created DiG-bench, a test of 70 text-based games measuring whether AI systems can figure out unstated rules by trying things out.
- Anthropic's Claude Opus 5 and a model called Fable 5 performed best. Most current leading AI models failed the hardest challenges.
- The benchmark targets a specific weakness: most AI systems excel when rules are explicit but struggle when they must infer rules through experimentation.
Where they differ
Import AIJack Clark
Researchers released DiG-bench, a benchmark of 70 text-based games designed to measure how well AI systems can infer hidden rules through exploration and interaction rather than being explicitly taught them. Opus 5 and Fable 5 performed best, with only these two models beating any tasks in the hardest tier, while most current frontier models cannot solve the most difficult challenges.
TLDR AITLDR editorial team
dig.bench is a benchmark containing 70 text-based games measuring whether agents can experiment to discover unknown rules. Humans can solve even the hardest games, while the best models struggle to beat games in the top tier.