Two new research papers explore the nuances of Large Language Model (LLM) behavior and prediction capabilities. The first paper, MedConstBench, introduces a benchmark for evaluating how LLMs handle medical ethics, particularly when their behavior is guided by a "constitution." It found that the impact of these constitutions is selective and model-dependent, leading to the proposal of an adaptive selector to optimize constitutional choices. The second paper investigates LLMs' ability to predict user decisions in human-agent dialogue, specifically in scenarios where users might break promises. The study found that LLMs are better at predicting decisions based on past behavior than on spoken words, and that their predictions do not fully capture the influence of agent personality or user traits. AI
IMPACT These studies highlight the need for more sophisticated evaluation methods for LLMs, focusing on ethical reasoning and predictive accuracy in complex human interactions.
RANK_REASON The cluster contains two academic papers published on arXiv detailing new benchmarks and studies on LLM capabilities.
- alphaXiv
- arXiv
- Beauchamp
- Big Tech
- CatalyzeX
- Connected Papers
- DagsHub
- Gotit.pub
- Hugging Face
- Litmaps
- LLMs
- MedConstBench
- MedDilemma
- medical ethics
- ScienceCast
- scite Smart Citations
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →
