Evaluating Commercial AI Chatbots as News Intermediaries
A new study evaluated six major AI chatbots on their ability to accurately report emerging news facts. While top models achieved over 90% accuracy on multiple-choice questions, their performance dropped significantly in free-response formats and particularly on questions with false premises. The research also highlighted a notable accuracy disparity across languages, with Hindi queries yielding lower results and indicating a bias towards English-language sources. AI
IMPACT Highlights critical limitations in AI news intermediaries, including regional bias and vulnerability to misinformation, impacting reliable information dissemination.