MAdam: Metric-Aware Multi-Objective Adam
Researchers have introduced MAdam, a novel wrapper for the Adam optimizer designed to improve multi-objective optimization in machine learning. MAdam addresses two key issues: a weighting mismatch where Adam's statistics dilute objective preferences, and a geometric mismatch where Adam's adaptive metric distorts objective alignment. By preconditioning the update direction, MAdam ensures the realized update is governed by the metric-conditioned objective, leading to consistent improvements across various applications like multi-task learning and medical imaging. AI
IMPACT Enhances multi-objective optimization techniques, potentially improving performance in complex machine learning tasks.