Sinusoidal Representation Network

Researchers propose SIREN-RoPE to enhance Transformer attention with learnable rotation space

Researchers have introduced SIREN-RoPE, a novel approach to enhance Transformer architectures by treating the rotation manifold of Rotary Positional Embeddings (RoPE) as a learnable, signal-conditioned space. This metho…