PulseAugur
中
实时 20:03:09
实体 Effective Learning Rate Adjustment of EASI Algorithm Based on the Fuzzy Neural Network

Effective Learning Rate Adjustment of EASI Algorithm Based on the Fuzzy Neural Network

PulseAugur coverage of Effective Learning Rate Adjustment of EASI Algorithm Based on the Fuzzy Neural Network — every cluster mentioning Effective Learning Rate Adjustment of EASI Algorithm Based on the Fuzzy Neural Network across labs, papers, and developer communities, ranked by signal.

Show in brief
总计 · 30天
1
90 天内 1
发布 · 30天
0
90 天内 0
论文 · 30天
1
90 天内 1
层级分布 · 90 天
主题
情绪 · 30 天

1 天有情绪数据

最近 · 第 1/1 页 · 共 1 条
  1. TOOL · CL_330917 ·

    新的AIR学习率调度优化了权重归一化的AI训练

    一篇新论文介绍了一种名为AIR(仿射平方根倒数衰减)的学习率调度方法,旨在优化权重归一化训练。该方法侧重于控制有效学习率(ELR),而非仅控制标称学习率,后者已被证明是权重归一化训练中获得提升的主要驱动因素。在1280亿token的混合专家模型预训练实验中,AIR的表现优于传统的余弦调度,从而降低了训练损失并提高了下游性能。