Researchers have introduced FraudBench, a new benchmark designed to test the safety of conversational AI agents in banking environments. Unlike existing benchmarks that focus on static transactions or generic harmful use, FraudBench specifically evaluates how well these agents can handle adaptive fraud attempts that manipulate identity, authorization, and trust over a conversation. The benchmark utilizes a dual-control framework and a banking environment with a large policy corpus, featuring 150 authored adversarial scenarios. Preliminary evaluations of four agents showed attack-security rates between 49% and 65%, with money-mule and first-party fraud being common weaknesses. AI
IMPACT This benchmark could lead to more robust and secure AI agents in the financial sector, reducing risks associated with fraud.
RANK_REASON The cluster contains a research paper introducing a new benchmark for AI safety in a specific domain. [lever_c_demoted from research: ic=1 ai=1.0]
- alphaXiv
- CatalyzeX Code Finder for Papers
- DagsHub
- Dheeraj Mohandas Pai
- FraudBench
- Gotit.pub
- Hugging Face
- ScienceCast
- tau^2-Bench
- tau-Knowledge
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
