Sierpiński triangle

LLMs struggle to maintain internal world models for complex planning tasks

Researchers have investigated why large language models struggle with planning puzzles like the Tower of Hanoi, particularly a variant where initial and goal states are complex. By training smaller Transformers on preco…