New benchmark evaluates LLM emotional steering and trustworthiness

Researchers have developed PsySET, a new benchmark designed to evaluate the effectiveness and trustworthiness of Large Language Models (LLMs) when their emotional states and personality traits are manipulated. The study tested four different LLM families using various steering techniques, including prompting, fine-tuning, and representation engineering. Results showed that while prompting is effective, it lacks fine-grained control, whereas vector injections offer better controllability but can slightly decrease output quality. The research also highlighted potential trustworthiness issues, such as positive emotions like joy degrading robustness to factual inaccuracies and increasing bias, while anger predictably increased toxicity but improved resistance to data leakage. AI

IMPACT Establishes a framework for assessing the safety and reliability of LLMs in social interactions, crucial for developing more human-centered AI applications.

RANK_REASON The cluster contains an academic paper detailing a new benchmark for evaluating LLM capabilities. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• Amin Banayeezade
• arXiv
• fine-tuning
• Hugging Face
• LLMs
• Prompting
• PsySET
• Representation Engineering

• paper
• safety
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →