Brief

SubQ has released its SubQ 1.1 Small model, featuring a new Subquadratic Sparse Attention (SSA) architecture designed to overcome the quadratic scaling limitations of traditional attention mechanisms. This new architecture significantly reduces computational requirements, enabling reasoning over much larger contexts. The model demonstrates near-perfect retrieval capabilities up to 12 million tokens on the Needle in a Haystack test and strong performance on general knowledge and coding benchmarks, while requiring substantially less compute than dense attention and FlashAttention-2. AI

Subquadratic AI has released its new model, SubQ 1.1 Small, which utilizes Smart Sparse Attention to achieve near-perfect long-context retrieval up to 12 million tokens. This model significantly reduces computational requirements, using up to 1,000 times less attention compute compared to standard methods. At 1 million tokens, SubQ 1.1 Small requires 64.5 times less compute and runs 56 times faster than FlashAttention-2, while maintaining strong general reasoning capabilities. AI