Researchers have developed a new multi-hop theory of mind evaluation for large language models called Triadic Werewolf. This game extends the traditional Werewolf game by introducing a "Jester" role with inverted win conditions, requiring models to reason across three opposing utility functions. In tests with GPT-4.1, DeepSeek-V3.1, and Llama 3.3 70B Instruct, the Jester role proved highly successful, winning 60-70% of games, while the Werewolf faction rarely exceeded 20%. Notably, GPT-4.1 struggled, often voting out the Jester prematurely, indicating a weakness in this complex multi-agent reasoning scenario. AI
IMPACT This new evaluation method could reveal deeper insights into LLM reasoning capabilities beyond current benchmarks.
RANK_REASON The cluster contains a research paper detailing a new evaluation method for LLMs.
Read on arXiv cs.MA (Multiagent) →
- DeepSeek-V3.1
- GPT-4.1
- Llama 3.3 70B Instruct
- theory of mind
- Triadic Werewolf
- jester
- Villagers
- werewolf
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →
