New Looped CED Architecture Slashes KV Cache Needs for Large Language Models

Researchers have developed a novel looped causal encoder-decoder (CED) architecture that significantly reduces the memory footprint associated with recurrent language models. This new design achieves recurrent depth without increasing the KV cache size across layers and rounds, a major bottleneck in current systems. By strategically placing recurrence in the encoder, the model iteratively refines global KV representations, leading to improved language modeling performance and long-context scaling. Testing on the 552B-parameter DeepSeek-V4.1-Flash model demonstrated a substantial reduction in KV cache usage, enabling significantly higher throughput and batch sizes during inference compared to unlooped models. AI

IMPACT This architectural innovation could significantly reduce inference costs and increase the efficiency of deploying large language models, enabling wider adoption and more complex applications.

RANK_REASON Academic paper detailing a new model architecture and its performance improvements. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• GB200
• Hugging Face
• KV cache
• Looped CED
• transformer

• paper
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →