New research explores capacity-redundancy trade-offs in multi-task learning

A new paper introduces the Capacity--Redundancy (CR) identity to analyze multi-task learning, proposing that negative transfer can stem from limited shared capacity and weak task redundancy. The research offers a clustering-gap decomposition for optimal sharing strategies and a gradient--TC bridge to link gradient similarity with redundancy ordering. Empirical results demonstrate that clustered LoRA significantly reduces residual coupling and outperforms random partitions, showing statistically significant performance gains. AI

IMPACT This research could lead to more efficient and effective multi-task learning models, potentially improving performance across various AI applications.

RANK_REASON The cluster contains a single academic paper published on arXiv. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• arXiv
• Capacity--Redundancy (CR) identity
• Hugging Face
• LoRA
• multi-task learning
• total correlation (TC)

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →