OPTScientist

New AI framework automates optimizer discovery for Transformer pretraining

Researchers have developed OPTScientist, a novel multi-agent framework designed to automate the discovery of optimizer programs for training deep learning models, particularly Transformers. This system employs a typed d…