Hard labels sampled from sparse targets mislead rotation invariant algorithms
A new research paper highlights a fundamental problem with how rotation-invariant algorithms learn from sparse target data in logistic regression. The study demonstrates that these algorithms can be provably suboptimal when dealing with hard labels sampled from a specific distribution, especially when the underlying weight vector is sparse. The paper proposes that non-rotation-invariant algorithms can achieve better performance by reparameterizing weights, offering a more efficient approach for learning noise-free soft targets. AI
IMPACT Identifies a theoretical limitation in common machine learning algorithms, potentially guiding the development of more robust learning methods.