GQA
PulseAugur coverage of GQA — every cluster mentioning GQA across labs, papers, and developer communities, ranked by signal.
11 day(s) with sentiment data
-
New TGIF module reduces hallucinations in multimodal LLMs
Researchers have developed TGIF (Text-Guided Inter-layer Fusion), a novel module designed to reduce hallucinations in multimodal large language models (MLLMs). Unlike previous methods that focus on text or static visual…
-
KV Cache Emerges as LLM Bottleneck, Driving Attention Variant Innovations
The KV cache, a critical component in autoregressive decoding for LLMs, is identified as the primary bottleneck for frontier models in 2026. Its size grows linearly with context length and batch size, making it the domi…
-
Liquid AI releases on-device agentic model LFM2.5-2.6B with 128K context
Liquid AI has released LFM2.5-2.6B, an open-weights, on-device agentic model designed for mobile and edge devices. This model boasts 2.69 billion parameters, a 128,000-token context window, and can perform multi-step ta…
-
Foveated probes reveal localized information in vision foundation models
Researchers have developed a new method called "foveated probes" to better assess the localized information retained within frozen vision foundation models. Unlike traditional global image embeddings, foveated probes us…
-
Frontier LLMs Distill Answer Set Programming Theories with High Accuracy
A new study explored distilling Answer Set Programming (ASP) theories from large language models using a neurosymbolic approach. The research tested nine models, including frontier models like Claude Sonnet 4.6, Claude …
-
Argus-Unified model offers economical image understanding and generation
Researchers have developed Argus-Unified, a novel unified multimodal model designed for both image understanding and generation. This model is notable for its compact size and economical training, utilizing a two-stage …
-
MAViE encoder boosts vision-language model efficiency by 80%
Researchers have introduced MAViE, a Multi-scale Adaptive Vision Encoder designed to improve the efficiency and effectiveness of vision-language models. MAViE utilizes position-dependent gates to integrate features from…
-
Grouped-Query Attention Optimizes LLM Inference by Reducing KV Cache Bottleneck
Grouped-query attention (GQA) is a technique that significantly improves the efficiency of large language models during inference by reducing the memory bottleneck caused by the KV cache. Unlike traditional Multi-Head A…
-
Moonshot AI releases Kimi-K3 model with long-context optimizations
Moonshot AI has released the Kimi-K3 model weights on Hugging Face, featuring architectural optimizations for long-context inference. The model employs a modified Transformer architecture with Grouped Query Attention (G…
-
New research optimizes transformer attention with Mathematics of Arrays
A new research paper details a method for optimizing transformer attention inference using the Mathematics of Arrays (MoA). The paper presents four memory-efficient artifacts, including a single-query decode DNF that al…
-
Visual token pruning impacts MLLM calibration, research finds
A new research paper investigates the impact of visual token pruning on the calibration of multimodal large language models (MLLMs). The study, published on arXiv, reveals that the method used for pruning tokens signifi…
-
LLM KV Cache Management: Security Threats and Efficiency Innovations
Researchers are exploring new methods to manage the Key-Value (KV) cache in large language models, which is crucial for inference speed but grows linearly with context length. One approach, "Error Certificates for KV-Ca…
-
Gemma-4 models ported to AWS Inferentia2 accelerators
The author successfully ported the entire Gemma-4 model family, including dense and Mixture-of-Experts (MoE) variants, to run on AWS Inferentia2 accelerators. This involved significant manual effort, as vendor-provided …
-
Modern LLM Transformer Blocks Evolve with RMSNorm, GQA, and MoE
Modern Transformer blocks in Large Language Models (LLMs) have evolved beyond the original 2017 design to improve training stability, context length, inference efficiency, and model capacity. Key advancements include th…
-
Evolution of Transformer Attention Mechanisms in Open-Source AI
The Transformer architecture's attention mechanism has seen significant evolution since its inception, with numerous advancements contributing to more efficient and capable large language models. Innovations like FlashA…
-
KV Cache Memory Explained: Estimating and Reducing VRAM Usage in LLMs
The KV cache, a critical component for LLM inference, can consume significant VRAM, often exceeding the memory required for model weights, especially at longer context lengths or higher batch sizes. A simple formula can…
-
PersistentKV optimizes LLM serving on commodity GPUs with new scheduling techniques
A new paper introduces PersistentKV, a system designed to optimize the serving of large language models (LLMs) with long contexts on commodity GPUs. PersistentKV employs page-aware decode scheduling and a native block-t…
-
Kamera method enhances multimodal AI efficiency with position-invariant KV cache
Researchers have developed a new method called Kamera that addresses the inefficiency of multimodal AI agents re-encoding information from repeated video frames or UI screenshots. This technique introduces a training-fr…
-
Grouped Query Experts enhance Transformer efficiency by selectively activating query heads
Researchers have introduced Grouped Query Experts (GQE), a novel mixture-of-experts layer designed to enhance the efficiency of Transformer models, particularly at long context lengths. GQE builds upon Grouped-Query Att…
-
xFormers library enables memory-efficient Transformer models on GPUs
This tutorial demonstrates how to build memory-efficient Transformer models using the xFormers library on GPUs. It covers implementing and comparing memory-efficient attention with standard attention, analyzing techniqu…