ruler
PulseAugur coverage of ruler — every cluster mentioning ruler across labs, papers, and developer communities, ranked by signal.
1 day(s) with sentiment data
-
New EXACT method boosts LLM long-context understanding
Researchers have developed a new supervision objective called EXACT to improve long-context adaptation in language models. This method addresses a mismatch in packed training by assigning extra weight to targets that re…
-
FocuSFT improves LLM long-context understanding via bilevel optimization
Researchers have developed FocuSFT, a novel bilevel optimization framework designed to improve how large language models handle long contexts. This method addresses the issue of "attention dilution," where models tend t…
-
New paper proposes residual-mass accounting for partial-KV decoding
Researchers have developed a novel method for partial-KV decoding, which optimizes the efficiency of large language models by only computing exact softmax contributions for a subset of tokens. This approach uses learned…
-
Subquadratic debuts 12M-token context window with linear scaling architecture
Subquadratic, a startup with 11 PhD researchers, has launched a new model featuring its Subquadratic Selective Attention (SSA) architecture, which claims to scale linearly with context length. This innovation allows for…
-
Q-RAG method enables efficient multi-step retrieval for LLMs up to 10M tokens
Researchers have introduced Q-RAG, a novel method for enhancing Retrieval-Augmented Generation (RAG) systems. This approach utilizes reinforcement learning to fine-tune the embedder model for multi-step retrieval, a mor…
-
Understanding and Improving Length Generalization in Hierarchical Sparse Attention Models
Researchers have identified three key design principles crucial for enhancing length generalization in hierarchical sparse attention models. These principles include using an expressive Chunk Encoder with a CLS token fo…
-
SIEVES method boosts multimodal LLM coverage on visual tasks with evidence scoring
Researchers have developed SIEVES, a novel method for improving the reliability of multimodal large language models (MLLMs) in out-of-distribution scenarios. SIEVES works by learning to estimate the quality of visual ev…