A study on the OLMo model found that filtering training data to remove undesirable traits often has minimal impact on the model's behavior. Researchers attempted to remove data points associated with specific behaviors like 'both-side framing' or 'liberal-lean' using various attribution methods, but these efforts were largely ineffective, performing similarly to random data removal. The only behavior that showed significant reduction through filtering was refusal, where probes and LLM judges proved most effective. The study suggests that many undesirable behaviors may already be present in the model's mid-training phase and are elicited rather than directly taught, with behaviors often bundled into personas. AI
IMPACT Suggests that current methods for controlling LLM behavior through data filtering are insufficient, potentially requiring new approaches to alignment.
RANK_REASON The cluster is based on a research paper detailing findings about LLM training data filtering. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
