New MELLON LLM boosts web navigation accuracy with multimodal inputs

Researchers have developed MELLON, a Multimodal Enhanced LLM for Online Navigation, designed to improve the performance of web navigation agents. This new approach focuses on aligning text and image inputs, enhancing multimodal reasoning and planning capabilities. In tests on the WebShop benchmark, MELLON achieved a 9.26% increase in task completion accuracy after a single training epoch, highlighting the potential of multimodal strategies for more effective web navigation. AI

IMPACT Enhances multimodal reasoning for web navigation agents, potentially improving user experience and task completion in online environments.

RANK_REASON Publication of a research paper detailing a new model and benchmark results. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• arXiv
• Multimodal Ranker
• VQAgent
• WebShop

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →