SubQ unveils SubQ 1.1 Small with 12M-token context and sparse attention

SubQ has released its SubQ 1.1 Small model, featuring a new Subquadratic Sparse Attention (SSA) architecture designed to overcome the quadratic scaling limitations of traditional attention mechanisms. This new architecture significantly reduces computational requirements, enabling reasoning over much larger contexts. The model demonstrates near-perfect retrieval capabilities up to 12 million tokens on the Needle in a Haystack test and strong performance on general knowledge and coding benchmarks, while requiring substantially less compute than dense attention and FlashAttention-2. AI

IMPACT This model's efficient attention mechanism could significantly lower the cost of training and inference for large-context LLMs, enabling new applications.

RANK_REASON New model release from a lab focused on frontier model architectures. [lever_c_demoted from frontier_release: ic=1 ai=1.0]

Read on Hacker News — AI stories ≥50 points →

• FlashAttention-2
• GPQA Diamond
• LiveCodeBench
• Needle in a Haystack
• NVIDIA
• SubQ 1.1 Small
• SubQ LLM
• Subquadratic Sparse Attention

• model release
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →