A new paper introduces HIVE (Human Input-Variation Engine), a tool designed to study how different input methods affect large language model (LLM) performance. The research found that voice transcription perturbations significantly reduce accuracy across tested models, with the structure of the transcription being more detrimental than filler words. Keyboard input perturbations, while less costly, also impact accuracy, with the number of original question tokens surviving the perturbation being the key factor in performance degradation. The study also noted that these issues are more pronounced when answers require construction or deduction, and that a "thinking budget" can recover keyboard channel performance but not voice registers. AI
IMPACT Investigating input methods can lead to more robust LLM agents and improved human-AI interaction.
RANK_REASON The cluster contains a research paper detailing a new methodology and findings on LLM input perturbations. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
