Proactive Conversational Assistant for a Procedural Manual Task based on Audio and IMU
Researchers have developed a novel conversational assistant capable of guiding users through procedural manual tasks using only audio and IMU data, bypassing the need for computationally intensive and privacy-compromising video input. This system proactively delivers step-by-step instructions and answers user queries, demonstrating improved performance through fine-tuning an existing language model. The fine-tuned model achieved a 50% increase in precision by reducing unnecessary dialogue and a 150% increase in recall for correct answers, with the entire system designed for edge device implementation without cloud dependency. AI
IMPACT This research could enable more private and efficient AI assistants for hands-on tasks, potentially reducing hardware costs and computational load.