Rotary Position Embedding
PulseAugur coverage of Rotary Position Embedding — every cluster mentioning Rotary Position Embedding across labs, papers, and developer communities, ranked by signal.
1 天有情绪数据
-
新的MLA注意力机制将LLM KV缓存削减高达10倍
多头潜在注意力（MLA）是一种新颖的注意力机制，旨在显著压缩大型语言模型的KV缓存。通过将KV对投影到低维潜在空间，MLA实现了大量的缓存缩减，使DeepSeek-V2/V3和Kimi K2.x等模型能够以更少的内存处理更长的上下文和更大的批次。该技术改变了前缀缓存和注意力计算的实现方式，在模型推理过程中提供了内存使用和计算成本之间更有效的权衡。
-
Transformer architecture explained: self-attention, RoPE, and FFNs
The Transformer architecture, introduced in the "Attention Is All You Need" paper, is fundamental to modern Large Language Models (LLMs). Key components include self-attention, which calculates token relationships, and …
-
AI researchers develop physics-informed transformer for universal building thermal models
Researchers have developed a physics-informed transformer architecture designed to create a universal thermal model for residential buildings. This model embeds domain knowledge and uses Rotary Position Embedding attent…
-
SHARP method enhances remote sensing image synthesis with dynamic resolution promotion
Researchers have developed SHARP, a novel method for enhancing the resolution of remote sensing images generated by diffusion models. SHARP fine-tunes the FLUX model on a large dataset of remote sensing imagery to creat…