Jordan Rope

Jordan-RoPE: Non-Semisimple Relative Positional Encoding via Complex Jordan Blocks

Researchers have introduced Jordan-RoPE, a novel relative positional encoding method for transformer models that utilizes complex Jordan blocks. This approach generates oscillatory-polynomial features, enabling a distan…