BABILong
PulseAugur coverage of BABILong — every cluster mentioning BABILong across labs, papers, and developer communities, ranked by signal.
1 day(s) with sentiment data
-
FocuSFT improves LLM long-context understanding via bilevel optimization
Researchers have developed FocuSFT, a novel bilevel optimization framework designed to improve how large language models handle long contexts. This method addresses the issue of "attention dilution," where models tend t…
-
New paper proposes residual-mass accounting for partial-KV decoding
Researchers have developed a novel method for partial-KV decoding, which optimizes the efficiency of large language models by only computing exact softmax contributions for a subset of tokens. This approach uses learned…
-
Q-RAG method enables efficient multi-step retrieval for LLMs up to 10M tokens
Researchers have introduced Q-RAG, a novel method for enhancing Retrieval-Augmented Generation (RAG) systems. This approach utilizes reinforcement learning to fine-tune the embedder model for multi-step retrieval, a mor…
-
Understanding and Improving Length Generalization in Hierarchical Sparse Attention Models
Researchers have identified three key design principles crucial for enhancing length generalization in hierarchical sparse attention models. These principles include using an expressive Chunk Encoder with a CLS token fo…