Automating SKILL.md Generation for Computer-Using Agents via Interaction Trajectory Mining
Researchers have developed a method to automatically generate SKILL.md files for computer-using agents by mining interaction trajectories. This three-stage pipeline segments GUI trajectories, clusters them into candidate skills, and trains a skill-aware policy. While the mined clusters show high purity against existing labels on a benchmark, they did not significantly improve downstream policy performance on metrics like GRPO and BrowseComp+. The study concludes that current methods for skill detection and representation are insufficient for reliable cross-domain policy improvement, despite revealing inspectable skill structures. AI
IMPACT This research highlights the challenges in translating mined agent skills into improved downstream policy performance, indicating areas for future development in agent training.