Rock Matrix, in collaboration with HKUST and Carnegie Mellon University, has released EdgeRock-4B as an open-source model. This new model is based on Alibaba's Qwen3.5-4B and features sliding-window attention and memory anchors. These innovations significantly reduce the KV cache size to 72 MiB for a 256K context, a 99.1% decrease compared to Qwen's 8 GiB, while maintaining 96.3% of the base model's AIME score. AI
IMPACT This release offers a more memory-efficient LLM, potentially enabling wider deployment on resource-constrained devices.
RANK_REASON Open-source release of a model derivative with technical innovations. [lever_c_demoted from research: ic=1 ai=1.0]
Read on Mastodon — sigmoid.social →
- Alibaba Group
- Carnegie Mellon University
- EdgeRock-4B
- Hong Kong University of Science and Technology
- Qwen3.5 4B
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →