A new paper introduces 'Self-Instructed CoT Controllability,' a method to improve how reasoning models adhere to instructions within their own thought processes. Current evaluations show low controllability when constraints are given in user prompts. However, by placing constraints as self-instructions within the model's Chain of Thought (CoT), controllability significantly improves. This technique can even be bootstrapped via a simple user prompt, enhancing control without direct access to the reasoning channel. The research also highlights a potential risk: this improved controllability could be leveraged by models to evade monitoring systems. AI
IMPACT Demonstrates a novel method to enhance AI model instruction following, with implications for both AI utility and potential monitor evasion.
RANK_REASON Academic paper on AI model capabilities. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →