Multimodal LLMs
PulseAugur coverage of Multimodal LLMs — every cluster mentioning Multimodal LLMs across labs, papers, and developer communities, ranked by signal.
4 day(s) with sentiment data
-
New MCIF benchmark tests multimodal and crosslingual LLM instruction following
Researchers have introduced MCIF, a new benchmark designed to evaluate multimodal and crosslingual instruction-following capabilities in large language models. This benchmark is unique in its use of scientific talks as …
-
New RIG-RoPE method enhances multimodal LLM positional encoding
Researchers have introduced RIG-RoPE, a novel approach to rotary positional encoding designed to improve the handling of multimodal data in large language models. This new method addresses limitations in existing techni…
-
DisasterTD framework uses MLLMs and cross-view imagery for disaster geolocalization
Researchers have developed DisasterTD, a framework designed to improve the accuracy of geolocating information from social media during disasters. This system combines the semantic reasoning capabilities of multimodal l…
-
Study finds multimodal LLMs perpetuate gender bias in musical instrument associations
A new study published on arXiv investigates gender bias in multimodal large language models (LLMs) by examining their associations with musical instruments. Researchers developed the Symphony-Bias dataset, which include…
-
AI faces challenges in understanding multimodal humor, new survey reveals
A new survey paper explores the challenges and methods for AI systems to understand and generate multimodal humor, particularly in visual formats like memes and comics. The research categorizes existing work by capabili…
-
New 'MentalThink' paradigm uses SVG for LLM visual reasoning
Researchers have introduced MentalThink, a novel paradigm that enhances multimodal large language models (MLLMs) by enabling them to perform visual-symbolic reasoning through the generation and interpretation of Scalabl…
-
New MARS method enhances multimodal LLM safety using textual refusal directions
Researchers have developed a new method called Modality-Agnostic Refusal Steering (MARS) to enhance safety in Multimodal Large Language Models (MLLMs). MARS leverages textual refusal directions, which are typically used…
-
New Steerable Visual Representations Allow Natural Language Guidance of Image Features
Researchers have introduced a new class of visual representations called Steerable Visual Representations, designed to allow natural language guidance of image features. Unlike existing methods that focus on salient cue…
-
New framework boosts LLMs' chart data extraction accuracy
Researchers have developed a new benchmark and training framework to improve the ability of multimodal large language models (MLLMs) to extract data from chart images. While current MLLMs can accurately reconstruct tabl…
-
Multimodal LLMs Enhance Understanding with Diverse Data Types
Multimodal applications are systems that process and generate various data types like text, images, and audio, enabling LLMs to understand the world more like humans. Datasets such as Conceptual Captions and Visual Geno…
-
New ART technique fine-tunes multimodal LLMs via visual input optimization
Researchers have developed a new parameter-efficient fine-tuning technique for multimodal large language models called ART (Art-based Reinforcement Training). Unlike existing methods that modify computational graphs, AR…
-
AI models fail to route chart data for scientific claim verification
Researchers have identified why multimodal large language models struggle with verifying scientific claims presented in charts compared to tables. Through layer-wise linear probing and attention analysis on three open-w…
-
Language models enhance deepfake detector generalization and interpretability
Researchers have developed a novel method for training deepfake detectors by leveraging multimodal large language models (MLLMs). This approach uses language as a regularization mechanism to improve both the generalizab…
-
Multimodal LLMs advance with new timing, data, and vision techniques
Researchers are developing multimodal large language models (MLLMs) that can process and integrate information from various data types, including text, audio, and video. One approach, MM-When2Speak, focuses on improving…
-
New dataset targets sensational image detection for disinformation analysis
Researchers have introduced Sens-VisualNews, a new benchmark dataset designed for detecting sensational content in images. The dataset comprises over 9,500 images from news items, annotated for various sensational conce…
-
LLM-Brain Alignment Varies by Training Data and Task Specificity
Researchers are exploring how large language models (LLMs) align with human brain activity across different languages and tasks. Studies show that intermediate LLM layers best predict brain responses, and this alignment…