Nyström attention matches full attention for stock prediction

Researchers have developed a novel approach called Nyström attention that matches the performance of full attention mechanisms in cross-sectional stock prediction tasks. This new method, which decomposes the inter-stock multi-head attention module, accounts for a significant portion of model parameters and predictive value. The study found that while learned attention is nearly uniform, forcing exact uniformity eliminates predictive capabilities, suggesting that low-rank approximations are key. Nyström attention, using a limited number of landmarks, achieves certified equivalence to full attention at a reduced computational cost. AI

IMPACT This research could lead to more efficient and effective AI models for financial forecasting by reducing computational costs.

RANK_REASON The item is an academic paper detailing a new method for attention mechanisms in machine learning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• alphaXiv
• arXiv
• CatalyzeX Code Finder for Papers
• Connected Papers
• CORE Recommender
• DagsHub
• Gotit.pub
• graph convolutional network
• Hugging Face
• IArxiv Recommender
• Litmaps
• long short-term memory
• MaSTer
• Nyström attention
• ScienceCast
• scite Smart Citations
• Tost

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →