ENTITY
Asynchronous Pipeline Parallelism
Asynchronous Pipeline Parallelism
PulseAugur coverage of Asynchronous Pipeline Parallelism — every cluster mentioning Asynchronous Pipeline Parallelism across labs, papers, and developer communities, ranked by signal.
Total · 30d
0
1 over 90d
Releases · 30d
0
0 over 90d
Papers · 30d
0
1 over 90d
TIER MIX · 90D
TOPICS
RECENT · PAGE 1/1 · 2 TOTAL
-
New research shows one-step gradient delay is not a barrier for LLM pretraining
A new research paper explores asynchronous pipeline parallelism for large-scale LLM pretraining, challenging the notion that gradient delay is an insurmountable barrier. The study demonstrates that the choice of optimiz…
-
Basis Rotation Improves LLM Training Efficiency by 81.7%
A new research paper introduces "basis rotation" to address gradient staleness in asynchronous pipeline parallelism for large-scale distributed training. The authors identify that basis misalignment between the Hessian …