Amirhossein Basareh

Transformer models' knowledge capacity grows exponentially with dimension, study finds

A new arXiv paper investigates the information storage capacity of transformer models. Researchers found that the ability of transformers to memorize factual knowledge grows exponentially with their dimension, meaning t…