New H2S Model Excels at Long-Context Understanding by Compressing Evidence

Researchers have introduced Highlight-Then-Summarize (H2S), a novel paradigm for long-context understanding in large language models. This approach first identifies relevant evidence from lengthy documents and then condenses it into a summary before generating a final answer. The H2S-Dataset, comprising over 6,600 examples, and H2S-RL, a reinforcement learning method, were developed to train this compress-then-reason behavior. Evaluations on H2S-Bench show that the H2S-14B model significantly outperforms other open-source models, achieving strong results in evidence selection, summary quality, and final answer accuracy within a constrained output budget. AI

IMPACT This method could improve LLM efficiency and accuracy in processing long documents by focusing on relevant evidence.

RANK_REASON This is a research paper detailing a new method and dataset for LLM long-context understanding. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• arXiv
• H2S-14B
• H2S-Bench
• H2S-Dataset
• H2S-RL
• Highlight-Then-Summarize
• Hugging Face
• Qwen3.8-27B

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →