ENTITY
z-loss
z-loss
PulseAugur coverage of z-loss — every cluster mentioning z-loss across labs, papers, and developer communities, ranked by signal.
Total · 30d
0
2 over 90d
Releases · 30d
0
0 over 90d
Papers · 30d
0
2 over 90d
TIER MIX · 90D
TOPICS
RECENT · PAGE 1/1 · 2 TOTAL
-
New research analyzes Z-loss backward geometry in language models
A new paper analyzes Z-loss, a technique used to stabilize language model training, from a backward-pass perspective. The research introduces a "backward-transport" view that separates the Z-loss source from the archite…
-
New method stabilizes LLM pretraining by addressing output embedding geometry
Researchers have introduced Output Embedding Centering (OEC) as a novel strategy to stabilize the pretraining of large language models (LLMs). This method addresses the issue of output logit divergence, a common trainin…