Chinese AI search reliability shows bias, accuracy gaps

A new study published on arXiv evaluated the reliability of AI systems and search engines in providing factual information within the Chinese web ecosystem. Researchers constructed a dataset from real Chinese search logs to compare nine systems, including traditional search engines and AI Overviews, on their ability to answer Yes/No factual questions. The findings indicate that while systems are similarly accurate when they provide definitive answers, they differ significantly in how often they offer such answers, with search engines being more decisive than models like Qwen-Max. A consistent bias was observed where all systems performed better on 'yes' queries than 'no' queries, and the study also identified regions with higher attention to health-related searches, suggesting potential risks for misinformation exposure. AI

IMPACT Highlights potential biases and reliability issues in AI-driven information access, particularly for non-English content.

RANK_REASON Academic paper analyzing AI system reliability. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• arXiv
• Geng Liu
• Qwen-Max

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →