Hierarchical Developmental Curriculum

Tiny Transformers achieve higher arithmetic accuracy with new training methods

Researchers have developed a method to improve arithmetic reasoning in small language models by using algorithmic scratchpads and a hierarchical curriculum. They found that proper data loading and linguistic pretraining…