Seesaw

Seesaw method accelerates LLM training by optimizing batch size and learning rate

Researchers have developed a new method called Seesaw to accelerate the training of large language models by optimizing the scheduling of batch sizes and learning rates. This approach theoretically demonstrates an equiv…