新的LoLA方法提升Transformer的内存和召回能力

报道来源 [3]

1. arXiv cs.CL TIER_1 English(EN) · Luke McDermott, Robert W. Heath Jr., Rahul Parhi · 2026-06-16 04:00

   LoLA: Low-Rank Linear Attention With Sparse Caching

   arXiv:2505.23666v3 Announce Type: replace Abstract: The per-token cost of transformer inference scales with context length, preventing its application to lifelong in-context learning. Linear attention is an efficient alternative that maintains a constant memory footprint, even on…

2. arXiv cs.CV TIER_1 English(EN) · Chunlu Li, Yixuan Pan, Bai Du, Zhenyuan Chen, Yanzhao Li, Hui Dong, Hui Wang, Zhiqiang Zou · 2026-06-16 04:00

   Training-free sparse attention based on cumulative energy filtering

   arXiv:2606.16317v1 Announce Type: new Abstract: Sparse attention accelerates Diffusion Transformers (DiTs) for video generation by computing only the important tokens while skipping the rest. The token selection strategy is key to balancing sparsity and accuracy. We formulate the…

3. arXiv cs.CV TIER_1 English(EN) · Zhiqiang Zou · 2026-06-15 07:26

   Training-free sparse attention based on cumulative energy filtering

   Sparse attention accelerates Diffusion Transformers (DiTs) for video generation by computing only the important tokens while skipping the rest. The token selection strategy is key to balancing sparsity and accuracy. We formulate the token filtering process as a dual-goal optimiza…