Transformer learns analytic number theory heuristic from elliptic curve data

Researchers have trained a two-layer transformer encoder to classify rational elliptic curves based on their rank, achieving over 99% accuracy using the first 128 normalized Frobenius traces. Through mechanistic interpretability techniques, they identified a sparse circuit of 20 MLP neurons sufficient for prediction, implementing a push-pull detector architecture. Notably, the model's learned input weights closely matched the Mestre-Nagao sum heuristic, indicating it learned a result from analytic number theory directly from the data. AI

IMPACT Demonstrates transformers' capability to learn complex mathematical heuristics, potentially opening new avenues for AI in theoretical sciences.

RANK_REASON The cluster contains an academic paper detailing novel research findings in machine learning applied to number theory. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• analytic number theory
• arXiv
• CLS attention
• Frobenius traces
• L(E,1)
• Mestre-Nagao Heuristic
• MLP neurons
• Pranav Venkata Konda
• transformers

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →