ENTITY
Query-Key Normalization
Query-Key Normalization
PulseAugur coverage of Query-Key Normalization — every cluster mentioning Query-Key Normalization across labs, papers, and developer communities, ranked by signal.
Total · 30d
1
2 over 90d
Releases · 30d
0
0 over 90d
Papers · 30d
1
2 over 90d
TIER MIX · 90D
TOPICS
SENTIMENT · 30D
1 day(s) with sentiment data
RECENT · PAGE 1/1 · 2 TOTAL
-
SwiftQK optimizes LLM training with efficient tensor parallelism
Researchers have developed SwiftQK, a new method for optimizing Query-Key Normalization (QK-Norm) in large-language models trained with Tensor Parallelism. This technique significantly reduces the communication overhead…
-
New papers analyze neural network grokking via spectral geometry
Two new arXiv papers explore the phenomenon of 'grokking' in neural networks, where models generalize only after memorizing training data. One paper proposes 'Low-Rank Decay' (LRD) as a spectral regularizer to improve g…