CTR-Sink: Attention Sink for Language Models in Click-Through Rate Prediction
Researchers have developed CTR-Sink, a new framework designed to improve language models' performance in click-through rate prediction tasks. This method addresses the challenge of applying language models to user behavior sequences, which differ structurally from natural language. CTR-Sink introduces "attention sinks" between discrete user actions to help the model focus on meaningful behavioral boundaries and relationships, thereby enhancing prediction accuracy. Experiments on industrial and open-source datasets demonstrate the framework's effectiveness. AI
IMPACT Enhances language model utility in recommendation systems by improving focus on user behavior sequences.