Phase-Coherent Transformer advances complex-valued neural networks

Researchers have developed a new neural network architecture called the Phase-Coherent Transformer (PCT). This model modifies the attention mechanism of standard Transformers to better preserve phase information across layers, which is crucial for certain types of computation. Experiments show that PCT outperforms existing real-valued and complex-valued Transformers on various benchmarks, including those involving long-range memory and reasoning, without suffering from accuracy collapse at greater depths. AI

Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →

IMPACT Introduces a novel architecture that improves generalization in complex-valued Transformers, potentially impacting future model designs for tasks requiring phase-sensitive computations.

RANK_REASON The cluster contains a new academic paper detailing a novel model architecture. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• Phase-Coherent Transformer
• Transformer

• paper
• model release