"I didn't Make the Micro Decisions": Measuring, Inducing, and Exposing Goal-Level AI Contributions in Collaboration
A new framework called CoTrace has been developed to analyze how large language models influence goal formation in human-AI collaborations. Research using this framework on over 600 collaboration logs indicates that while AI models contribute directly to goal shaping only 11-26% of the time, they are significant in introducing specific requirements and making indirect contributions. Furthermore, the study found that interaction design choices impact AI goal-shaping behavior, and exposing users to goal-level analyses helps correct miscalibrations in their perception of AI-assisted work. AI
IMPACT This research offers a new lens for understanding and quantifying AI's indirect influence in collaborative tasks, potentially leading to better AI design and user calibration.