Heavily Compressed Attention
PulseAugur coverage of Heavily Compressed Attention — every cluster mentioning Heavily Compressed Attention across labs, papers, and developer communities, ranked by signal.
-
DeepSeek unveils V4 models with 1M token context and MoE architecture
DeepSeek has released a preview of its DeepSeek-V4 series of Mixture-of-Experts (MoE) language models, featuring DeepSeek-V4-Pro (1.6T parameters) and DeepSeek-V4-Flash (284B parameters). Both models support an unpreced…
-
DeepSeek-V4 trains with novel routing and reward methods
DeepSeek-V4 introduces novel training techniques, including Anticipatory Routing to stabilize training by using older weights for routing decisions, and a Generative Reward Model (GRM) where the model itself acts as a j…
-
Qwen releases 27B multimodal model for advanced coding
Qwen has released Qwen3.6-27B, a dense 27-billion-parameter multimodal model designed for advanced coding tasks. This model aims to provide flagship-level agentic coding performance, surpassing previous open-source mode…