Gemma 4-12B model uses shared numerical concept across modalities

Researchers have investigated how multimodal AI models process numerical information across different input types and operations. Using causal interventions on Gemma 4-12B, they identified distinct coordinate frames for text and image inputs, which can be aligned through rotation and rescaling. The study also revealed that a shared numerical concept is formed upstream and then unfolded over time, with hidden internal directions carrying the full numerical result even when not directly visible to the output layer. AI

IMPACT Provides insight into how multimodal models represent and process numerical concepts, potentially informing future model architectures.

RANK_REASON Academic paper detailing novel research findings on AI model internal representations. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• Gemma 4-12B
• Xavier Suau Cuadros

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →