Jacobian Rank Collapse identified in decision-focused learning

Researchers have identified a phenomenon called Jacobian Rank Collapse in decision-focused learning (DFL), where the predictor's Jacobian matrix exhibits low rank. This condition means that the gradients used by the optimizer are collinear, limiting the independent parameter-update directions available for learning. Experiments across various configurations, including equity, shortest-path, and knapsack problems, show that while DFL can offer modest improvements in decision quality, these gains are often marginal and can be sensitive to factors like coordinate scaling and predictive accuracy versus decision quality. AI

IMPACT Identifies a geometric limitation in decision-focused learning that may affect its practical benefits.

RANK_REASON Academic paper on a novel concept in machine learning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• Decision-Focused Learning
• Hugging Face
• Jacobian Rank Collapse
• SGD

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →