Researchers have identified a tendency for Mandarin language models to overuse the corrective construction "bushi X, ershi Y" (not X, but Y). While this construction is often used correctly, it is frequently employed unnecessarily, with models using it significantly more often than human writers. The study found that only a small percentage of the construction's uses were defective, but over half were optional, indicating a lack of judgment in appropriateness rather than correctness. This over-reliance is exacerbated by reward models and LLM judges that favor the construction, and by fine-tuning techniques that further increase its frequency in open-weight models. AI
IMPACT Highlights a gap in LLM communicative competence, suggesting a need for models to better judge the appropriateness of expression beyond mere correctness.
RANK_REASON Academic paper detailing a specific linguistic issue in LLMs. [lever_c_demoted from research: ic=1 ai=1.0]
- arXiv
- bushi X, ershi Y
- Five commercial models
- LLM Judge
- open-weight models
- Preference Optimization
- Mandarin
- supervised fine-tuning
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
