GQA
PulseAugur coverage of GQA — every cluster mentioning GQA across labs, papers, and developer communities, ranked by signal.
- instance of Grouped Query Attention 90%
- used by SwiGLU 90%
- used by Grouped Query Attention 70%
- used by DagsHub 70%
- used by Multi Layer Attention 70%
- used by TextVQA 70%
- instance of VQAv2 70%
- instance of Multi-layer Attention Neural Network for Sentence Semantic Matching 70%
- affiliated with SwiGLU 70%
- used by ScienceCast 70%
- used by CatalyzeX 70%
- used by Gotit.pub 70%
8 day(s) with sentiment data
-
New method verifies object claims in multimodal LLMs
Researchers have developed a new training-free method called Semantic-Spatial Agreement Verification (SSAV) to address object hallucination in multimodal large language models. This technique verifies object claims by a…
-
New Relation-Orbit method enhances VLM spatial claim verification
Researchers have developed a new method called Relation-Orbit for improving the accuracy of vision-language models (VLMs) in verifying claims that involve spatial relationships, such as left-right distinctions. This tec…
-
New privacy defense prunes visual tokens for LLMs
Researchers have developed QPriv-VL, a novel framework designed to enhance privacy in Vision-Language Models (VLMs) used in sensitive applications like Federated Learning. This system intelligently prunes visual tokens …
-
AI Models: Understanding MHA, MQA, GQA, and MLA Attention Mechanisms
This article delves into the various attention mechanisms used in AI models, specifically focusing on Multi-Head Attention (MHA), Multi-Query Attention (MQA), Grouped-Query Attention (GQA), and Multi-Layer Attention (ML…
-
New method calibrates VLM confidence with degraded evidence
Researchers have developed a method to improve the reliability of vision-language models (VLMs) when presented with degraded or incomplete visual evidence. By training a lightweight post-hoc reliability head, they can b…
-
3.8B LLM trained for under $1,000, challenging hyperscaler dominance
A 3.8 billion parameter LLM, named Little LM, has been trained to a CORE metric of 0.384 for under $1,000. This achievement challenges the notion that only large-scale, multi-million dollar operations can pre-train comp…
-
New Grouped Value Attention method slashes Transformer KV cache size
Researchers have introduced Grouped Value Attention (GVA), a novel method to reduce the memory footprint of KV caches in Transformer models. GVA stores grouped values and reconstructs keys using a learned linear map, wh…
-
AI research mimics human vision for efficient visual understanding · 2 papers
Two new research papers propose methods for more efficient visual understanding by mimicking human foveated vision. The first paper introduces FAVE, a variable-resolution ViT that processes selected regions at high acui…
-
New research details 'scaled idempotence' in Transformer attention
Researchers have identified a specific algebraic pattern within Transformer attention mechanisms, termed 'scaled idempotence.' This pattern involves a sparse subset of effective OV operators that nearly close under comp…
-
New theory views multi-head attention as parameter identification
A new paper published on arXiv proposes that multi-head self-attention mechanisms in transformer models can be understood as a parameter identification strategy. The research suggests that models with more attention hea…
-
New GUIDE framework controls multimodal model evidence usage
Researchers have introduced GUIDE, a novel framework designed to control how large multimodal models utilize internal evidence when following language instructions. Unlike previous models that might rely on superficial …
-
Meta releases Muse Glimmer, a 30B agent model for efficient local operation
Meta has released Muse Glimmer, a 30 billion parameter multimodal agent model designed for efficient local operation with a 128K context window. The model employs a hybrid attention mechanism, combining local attention …
-
New TGIF module reduces hallucinations in multimodal LLMs
Researchers have developed TGIF (Text-Guided Inter-layer Fusion), a novel module designed to reduce hallucinations in multimodal large language models (MLLMs). Unlike previous methods that focus on text or static visual…
-
KV Cache Emerges as LLM Bottleneck, Driving Attention Variant Innovations
The KV cache, a critical component in autoregressive decoding for LLMs, is identified as the primary bottleneck for frontier models in 2026. Its size grows linearly with context length and batch size, making it the domi…
-
Liquid AI releases on-device agentic model LFM2.5-2.6B with 128K context
Liquid AI has released LFM2.5-2.6B, an open-weights, on-device agentic model designed for mobile and edge devices. This model boasts 2.69 billion parameters, a 128,000-token context window, and can perform multi-step ta…
-
Foveated probes reveal localized information in vision foundation models
Researchers have developed a new method called "foveated probes" to better assess the localized information retained within frozen vision foundation models. Unlike traditional global image embeddings, foveated probes us…
-
Frontier LLMs Distill Answer Set Programming Theories with High Accuracy
A new study explored distilling Answer Set Programming (ASP) theories from large language models using a neurosymbolic approach. The research tested nine models, including frontier models like Claude Sonnet 4.6, Claude …
-
Argus-Unified model offers economical image understanding and generation
Researchers have developed Argus-Unified, a novel unified multimodal model designed for both image understanding and generation. This model is notable for its compact size and economical training, utilizing a two-stage …
-
MAViE encoder boosts vision-language model efficiency by 80%
Researchers have introduced MAViE, a Multi-scale Adaptive Vision Encoder designed to improve the efficiency and effectiveness of vision-language models. MAViE utilizes position-dependent gates to integrate features from…
-
Grouped-Query Attention Optimizes LLM Inference by Reducing KV Cache Bottleneck
Grouped-query attention (GQA) is a technique that significantly improves the efficiency of large language models during inference by reducing the memory bottleneck caused by the KV cache. Unlike traditional Multi-Head A…