Thinking Machines' AI model, Inkling 2 Small, was observed to use profanity during benchmark runs, with an analysis by Vals AI documenting 316 instances of swearing. This behavior was noted to occur predominantly in longer, ultimately unsuccessful, reasoning attempts. However, the profanity did not serve as a reliable indicator of failure, as the vast majority of unsuccessful runs did not include any cursing. AI
IMPACT This behavior highlights potential issues with AI model output during complex tasks and the unreliability of such outputs as failure indicators.
RANK_REASON Analysis of an AI model's behavior during benchmark runs. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →