Parity
PulseAugur coverage of Parity — every cluster mentioning Parity across labs, papers, and developer communities, ranked by signal.
-
New metric reveals transformer length generalization limits
Researchers have developed a method called normalized exact-solution volume (NESV) to analyze the length generalization capabilities of transformers. This approach quantifies the fraction of a transformer's parameter sp…
-
Transformer parameter space geometry hinders learning of sensitive functions
Researchers have identified a key reason why transformer models struggle to learn certain functions, such as PARITY. The study reveals that even when these functions are representable by transformers, the specific param…
-
AI theory paper shows transformers need at least two layers for parity task
Researchers have demonstrated that a two-layer transformer model is the minimum required to compute the PARITY task, which determines if a binary sequence has an even or odd number of ones. They proved that a one-layer …