SinkRec: Mitigating Semantic State Sink in Long Sequence Recommendation with Memory-Conditioned Gated Delta Networks
Two new research papers address challenges in training recommendation models with extremely long user interaction histories. The first, "Versioned Late Materialization," proposes a system to reduce data infrastructure load by storing user history once and reconstructing sequences on demand, enabling longer sequences and improving model quality. The second paper, "SinkRec," introduces a hybrid memory-transition architecture to mitigate "semantic state sink" in linear attention models, preventing repetitive patterns from overwhelming the model's state and improving efficiency for long sequences. AI
IMPACT These methods aim to improve the efficiency and effectiveness of recommendation systems by enabling them to process longer user interaction histories.