DeepNorm

DeepLoop method enhances Looped Transformer depth scaling

Researchers have introduced DeepLoop, a novel method for scaling the depth of Looped Transformers. This technique addresses the challenge of residual scaling in recurrent architectures by formalizing the tied-depth effe…