LLM geometry enables cross-model steering transfer above 1.7B parameters

Researchers have conducted a systematic study on cross-architecture steering transfer in language models, demonstrating that shared internal representations of semantic concepts can be functionally exploited across different models. This transfer is contingent on the models' representational capacity, with a notable discontinuity observed around 1.7 billion parameters. Models at or above this scale showed significant alignment in feature pairs, enabling cross-model behavioral control without fine-tuning, whereas smaller models exhibited degraded transfer capabilities. The findings highlight the importance of scale thresholds in mechanistic interpretability, suggesting that tools validated on larger models may not directly apply to smaller ones. AI

IMPACT Establishes functional exploitability of shared LLM geometry, highlighting scale thresholds for interpretability tools.

RANK_REASON This is a research paper detailing empirical study of LLM properties. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• Cross-Architecture Steering Transfer
• Language Models
• Platonic Representation Hypothesis

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →