CLAP
PulseAugur coverage of CLAP — every cluster mentioning CLAP across labs, papers, and developer communities, ranked by signal.
3 day(s) with sentiment data
-
New C++ Tool Enables Audio Embedding Inference with GGML Models
A new C++ implementation called clap.cpp has been developed for audio embedding inference using GGML models. This tool supports various audio models like CLAP, CLAP2, CLaMP3, MERT, MuQ, and MuQ-MuLan, and can process WA…
-
New geometric approach closes modality gap in multimodal AI models
Researchers have proposed a novel geometric approach to bridge the modality gap in multimodal contrastive models like CLIP and CLAP. By shifting the embedding domain from a unit sphere to a probability simplex, the mode…
-
Frontier AI models struggle to label ASMR audio, human input sought
A project training a text-to-ASMR model found that current frontier AI models struggle with accurately labeling audio clips for specific sounds like paper rustling or brushing. While models like Gemini 3.1 Pro performed…
-
New Audit Method Reveals Model-Dependent Audio Grounding in LALMs
Researchers have developed a novel black-box causal audit to investigate hallucinations in Large Audio-Language Models (LALMs). This method isolates and removes evidence regions to test model reliability, revealing that…
-
Audio embeddings retain significant musical content, study finds
Researchers have investigated the information retained within audio embeddings generated by various pre-trained audio encoders. By using a shared latent diffusion decoder to reconstruct audio from these embeddings, they…
-
AI model fine-tuned for autism auditory hypersensitivity research
A doctoral research project is exploring audio emotion recognition for individuals with atypical hearing, specifically focusing on auditory hypersensitivity in people with autism. The approach involves fine-tuning the C…
-
New SonicCaps dataset enhances audio-language models with diverse captions
Researchers have introduced SonicCaps, a new large-scale dataset designed to improve audio-language modeling and audio retrieval. This dataset features approximately 15 million captions paired with 700,000 audio clips, …
-
LLM-Augmented Alignment Improves Zero-Shot Respiratory Sound Classification
Researchers have developed a novel framework for zero-shot respiratory sound classification by aligning self-supervised respiratory encoders with clinical terminology. This method utilizes a medical LLM to generate stru…
-
LLM Hallucination Signal Identified as Simple Mean Shift
Researchers have identified that the primary signal for detecting hallucinations in large language models is a simple mean shift within the model's hidden states. Across various 7B-scale models and datasets, complex pro…
-
New method audits generative audio LLMs, finds limited value in generative calls
Researchers have developed a new method to evaluate generative audio large language models (LLMs) by auditing their generative calls. This approach distinguishes between the value of acoustic evidence and the necessity …
-
New frameworks enhance embodied AI simulation with cross-embodiment learning and procedural generation
Researchers have developed new frameworks for creating more realistic and versatile simulation environments for embodied AI. CLAP enables training on diverse video data across different robotic and human agents by recon…
-
AI system uses multi-modal data for conversational music recommendations
Researchers from Team Semiintelligencn have developed a multi-modal system for conversational music recommendation, utilizing a three-stage pipeline for the ACM RecSys 2026 TalkPlayData Challenge. The system integrates …
-
New generative framework creates realistic synthetic populations for travel models
Researchers have developed a novel two-stage generative framework to create realistic synthetic populations for activity-based travel demand models. The first stage uses a regularized Wasserstein GAN with gradient penal…
-
New research tackles audio-visual event perception in language models
Two new research papers introduce novel approaches to audio-visual event perception in language models. The first, ST-OmniQA, presents a benchmark for spatio-temporal audio-visual reasoning with moving sound sources, al…
-
Gemini-3.1-Pro-Preview leads audio classification benchmark, outperforming competitors
A new benchmark evaluates eleven audio classification methods, including several Gemini models and Kimi-Audio-7B-Instruct, on a sound source identification task. The best performing model, Gemini-3.1-Pro-Preview, achiev…
-
CLAP model enables audio abuse detection in low-resource Indic languages
Researchers have developed a method for detecting audio abuse in low-resource Indic languages by leveraging the CLAP model. This approach bypasses the need for accurate speech-to-text transcription, which is often unrel…
-
Audio foundation models capture phylogenetic signal without domain-specific training
A new study published on arXiv investigates whether large audio foundation models can capture phylogenetic signals from species vocalizations without explicit training for this purpose. The research found that general-p…
-
New FuSiLi method aligns multimodal music data with global supervision
Researchers have developed FuSiLi (Fused Sinkhorn-Localized Similarity), a novel method for multimodal contrastive learning in music. This approach effectively learns localized relationships between audio and visual rep…
-
New methods enhance VLM to VLA adaptation for robotics control · 2 sources tracked
Two new research papers propose methods to improve the adaptation of vision-language models (VLMs) into vision-language-action (VLA) models for robotics. The first paper introduces CLAP (Causal Language-Action Predictio…
-
CLAP audio embeddings reliably encode acoustic attributes, study finds
Researchers have developed a probing framework to analyze CLAP audio embeddings, revealing how fundamental perceptual attributes are encoded within these representations. The study found that reverberation (RT60), loudn…