DeRes: Decoupling Residual Stability and Adaptivity for Scalable CTR Prediction
Researchers have introduced DeRes, a novel architecture for Transformer-based CTR prediction models that decouples residual stability and adaptivity. This new design employs parallel identity and block attention residual paths, allowing for better preservation of early signals and more effective recall of long-range dependencies. DeRes demonstrates superior performance on large-scale datasets, outperforming existing models with minimal additional computational cost and offering a significantly steeper compute-AUC scaling law. AI
IMPACT Introduces a more efficient architecture for CTR prediction, potentially improving recommendation systems and targeted advertising.