Expand More, Shrink Less: Shaping Effective-Rank Dynamics for Dense Scaling in Recommendation
Researchers have introduced RankElastor, a new architecture designed to address embedding collapse in dense recommendation models. This phenomenon, where learned representations have a low effective rank, limits model expressivity. RankElastor incorporates parameterized full mixing and GLU-improved feedforward networks to stabilize representation spectra and mitigate collapse. Experiments on large-scale industrial datasets show that RankElastor improves recommendation performance and scaling behavior. AI
IMPACT Introduces a novel architecture to improve the performance and scaling of recommendation systems by addressing a key technical challenge.