Researchers have developed a new method using sparse crosscoders to track the emergence and consolidation of linguistic features within large language models during pretraining. This technique, which includes a novel metric called Relative Indirect Effects (RelIE), helps identify when specific capabilities become causally important for task performance. The approach is architecture-agnostic and scalable, offering a more interpretable way to analyze representation learning in LLMs. Separately, another study explores the use of LLMs to detect language ideologies in Luxembourgish news comments, a small language with limited representation in training data. The research investigates whether machine translation to high-resource languages improves LLM performance on this task, suggesting LLMs can be practical tools for identifying ideological content despite current optimization limitations. AI
IMPACT Provides new methods for understanding LLM internal representations and explores LLM utility for sociolinguistic analysis.
RANK_REASON This cluster contains two academic papers published on arXiv, one detailing a new method for analyzing LLM pretraining and another exploring LLM applications in sociolinguistics.
AI-generated summary · Google Gemini · from 4 sources. How we write summaries →
