Large language models show surprising night sky map representations

Researchers have discovered that large language models, specifically those around 100 billion parameters, can encode a representation of the night sky map. This representation is decodable from the model's residual stream and often surfaces when prompts ask about celestial object proximity. In most tested open-source models, this sky sphere representation explained a significant portion of variance, with median angular errors as low as 12-21 degrees. The study confirms this is not a simple leak from a flat representation and identifies it as the first known example of a curved, high-dimensional irreducible feature manifold. AI

IMPACT Reveals unexpected emergent capabilities in large language models, suggesting deeper understanding of spatial and astronomical concepts.

RANK_REASON Academic paper detailing a novel finding about internal model representations. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• Alexander Berdnikov
• arXiv
• Hugging Face
• Language Models
• LOO testing
• open source models
• Sky sphere

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →