New MLLM framework unifies surgical scene understanding

Researchers have developed SurgMLLM, a novel framework that unifies surgical scene understanding by integrating high-level reasoning with low-level visual grounding. This multimodal large language model (MLLM) is fine-tuned to process surgical videos, enabling it to jointly model procedural phases, instrument-verb-target triplets, and their precise segmentation. The system achieved significant improvements on the CholecT45-Scene dataset, boosting the triplet recognition metric AP_IVT from 40.7% to 46.0% and outperforming existing methods in phase recognition and segmentation. AI

影响 Enhances AI's capability in medical procedures by enabling more comprehensive understanding of surgical videos.

排序理由 The cluster contains a research paper detailing a new framework and model for surgical scene understanding. [lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.AI 阅读 →

• CholecT45-Scene
• MLLM
• SurgMLLM

• 论文
• 模型发布

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →