AdaLeZO

AdaLeZO speeds up LLM fine-tuning with adaptive layer sampling

Researchers have developed AdaLeZO, a new framework designed to make Zeroth-Order (ZO) optimization more efficient for fine-tuning Large Language Models. This method addresses the slow convergence and high variance typi…