Flatness Preference Optimization

New PEFT method targets 'flatness preference' for better generalization

Researchers have identified a "flatness preference" in parameter-efficient fine-tuning (PEFT) methods, suggesting that a small subset of dimensions significantly impacts generalization. They propose Flatness Preference …