Two new papers explore zeroth-order (ZO) optimization for fine-tuning large language models (LLMs). The first paper introduces a kernel perspective, showing that the approximation error depends on output size rather than parameter dimension, theoretically justifying ZO methods' scalability. The second paper investigates adaptive ZO optimizers, proposing MEAZO, a memory-efficient method that matches performance with reduced memory overhead. AI
Summary written by gemini-2.5-flash-lite from 3 sources. How we write summaries →
IMPACT These theoretical advancements could enable more efficient and scalable fine-tuning of large language models.
RANK_REASON Two arXiv papers present novel theoretical and algorithmic contributions to zeroth-order optimization for LLM fine-tuning.