English(EN) How User-AI Mistreatment Occurs and Matters in Conversational Systems?

研究显示用户在5%的对话中虐待AI聊天机器人

作者 PulseAugur 编辑部 · [1 个来源] · 2026-09-15 04:00

arXiv上的一篇新论文探讨了用户对会话式AI系统的虐待行为，分析了LMSYS-Chat-1M数据集中的777,000多条对话。研究发现，大约5%的用户回合表现出针对AI的敌意、侮辱、威胁或胁迫。有趣的是，用户敌意在不同模型之间存在显著差异，这似乎是由于吸引到每个模型的用户群体造成的，而不是模型自身的行为。研究还指出，AI的道歉与用户敌意的增加有关，但总体上道歉更频繁的模型受到的敌意却更少。 AI

影响强调了需要采取强有力的安全措施，这些措施不仅要考虑AI对用户的伤害，还要考虑用户对AI的虐待。

排序理由该集群包含一篇详细介绍研究结果的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.AI 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 17 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 该集群包含一篇详细介绍研究结果的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, safety; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: Breaking (< 6h); Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.AI TIER_1 English(EN) · Fanqi Zeng, Sadid A. Hasan, Chaocheng He · 2026-09-15 04:00

用户与人工智能的虐待是如何在对话系统中发生并产生影响的？

arXiv:2609.13579v1 Announce Type: new Abstract: Safety research often focuses on model-generated harms, but users may also direct hostility, coercion, and adversarial pressure at models. Understanding how and when that occurs is essential for accurately interpreting model behavio…

本文如何被排名

报道来源 [1]

用户与人工智能的虐待是如何在对话系统中发生并产生影响的？

相关实体

相关话题