ML practitioner maps AI model circuits using contrastive SFT

A machine learning practitioner is exploring a novel method for understanding and controlling AI model behavior by mapping causal dependencies between different capabilities. The approach involves using contrastive supervised fine-tuning (SFT) to isolate specific circuits within a 31B parameter model. By training variants that emphasize or de-emphasize certain dimensions and then ablating identified circuits, the practitioner aims to build a causal dependency graph of model capabilities. This graph could then inform optimal training orders for future model development and enhance behavioral control. AI

IMPACT This research could lead to more predictable and controllable AI behavior by mapping internal causal dependencies.

RANK_REASON The item describes a novel research methodology for understanding AI model internals, not a formal publication or a product release. [lever_c_demoted from research: ic=1 ai=1.0]

Read on r/MachineLearning →

• 31B model
• Substantial_Diver469

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →