GQA
PulseAugur coverage of GQA — every cluster mentioning GQA across labs, papers, and developer communities, ranked by signal.
- used by SwiGLU 90%
- instance of Grouped Query Attention 90%
- instance of DagsHub 70%
- instance of Multi-layer Attention Neural Network for Sentence Semantic Matching 70%
- used by DagsHub 70%
- instance of ScienceCast 70%
- instance of CatalyzeX 70%
- instance of Gotit.pub 70%
- used by Multi Layer Attention 70%
- used by CatalyzeX 70%
- used by Gotit.pub 70%
- instance of VQAv2 70%
11 day(s) with sentiment data
-
DepthGraft architecture enhances LLM performance via hierarchical KV sharing
Researchers have introduced DepthGraft, a novel hierarchical KV-sharing architecture designed to improve the performance of large language models by addressing information dilution in deep layers. This method reconstruc…
-
New Latent-Lens method enables small language models to perceive visual data
Researchers have developed Latent-Lens, a novel method for enabling small language models (SLMs) to perceive visual information without direct training. This technique uses a lightweight codec to bridge the representati…
-
LLM power capping ineffective during decode, new research finds
A new research paper challenges the effectiveness of power capping on GPUs for Large Language Model (LLM) serving, particularly during the autoregressive decode phase. The study found that power caps have minimal impact…
-
llama.cpp project releases multiple pre-release updates with performance improvements
The llama.cpp project has released several pre-release versions, including b12121, b12119, b12118, b12116, b12114, b12110, b12109, b12102, b12100, and b12109. These updates address various issues and introduce new featu…
-
LLM KV Cache Sizing Guide for Local Models
This article explains how to accurately size the KV cache for local LLMs, which is crucial for preventing out-of-memory errors and performance degradation. It details the formula for calculating KV cache size based on m…
-
Fed-Universe architecture uses LLM for negotiated alignment in federated systems
Researchers have introduced Fed-Universe, a novel architecture designed to improve alignment in federated learning systems by moving beyond simple averaging. This Semantic-Geometric-Topological-Human (S-G-T-H) stack use…
-
vLLM KV Cache Memory Limits GPU Concurrency, Not Compute
A technical deep-dive into vLLM's KV cache reveals that memory, not compute, is the primary bottleneck for concurrent requests on GPUs. The KV cache, which stores attention keys and values for each token, grows linearly…
-
llama.cpp adds Nemotron support; KRAFTON AI previews Raon-1 mini model
The llama.cpp project has released version b11819, which includes support for non-standard GQA, BPE tokenizers, and domyn-small pre-types, specifically for the Nemotron model. This update also refines GGUF metadata hand…
-
LLM KV Cache Memory Usage Explained: MQA and GQA Techniques
The KV cache, essential for fast LLM generation, consumes significant GPU memory by storing keys and values for each token. Techniques like Multi-Query Attention (MQA) and Grouped-Query Attention (GQA) reduce this memor…
-
New method tackles relational OOD errors in vision-language models
Researchers have introduced a novel evaluation method called counterfactual pairing cycles to address the challenge of assessing relational out-of-distribution errors in vision-language models. This technique aims to pr…
-
Hacktoberfest AI Challenge: Submissions Highlight Gemma, Agent Sessions, and LLM Performance
Multiple sources are discussing submissions for the Hacktoberfest Open-Source AI Challenge, specifically focusing on Week 2: Side Hustle. These submissions highlight various aspects of AI development, including the use …
-
New Sparse Attention Method Boosts LLM Long-Context Decoding Throughput
Researchers have developed Joint Subspace Sparse Attention (JSSA), a novel method to improve the efficiency of long-context language model decoding. JSSA addresses challenges in token-level selection by learning a share…
-
New JSSA method boosts LLM long-context decoding speed by 6.43x
Researchers have developed Joint Subspace Sparse Attention (JSSA), a novel method to improve the efficiency of long-context decoding in large language models. JSSA addresses challenges in token-level selection by learni…
-
Virtual Head Attention improves LLM efficiency and quality · 2 sources tracked
Researchers have introduced Virtual Head Attention (VHA), a novel mechanism designed to overcome the "impossible triangle" challenge in long-context large language models. VHA aims to simultaneously improve KV cache siz…
-
New CHASE framework enhances AI model engineering with geometry-aware methods
Researchers have introduced CHASE (Channel-Aligned Structure Exploitation), a novel framework for geometry-aware model engineering. CHASE leverages Geometric and Spectral Alignment (GSA) to analyze trained neural networ…
-
New methods accelerate LLM decoding with sparse attention and pruning techniques · 5 sources tracked
Researchers are developing new methods to accelerate Large Language Model (LLM) inference, primarily focusing on optimizing the decoding stage which is often memory-bound. One approach, SparseDecoding, uses decoding-awa…
-
New NAMOH sparse attention mechanism enables efficient context scaling in LLMs
Researchers have introduced NAMOH, a novel sparse attention mechanism designed to improve the efficiency and effectiveness of large language models, particularly at extended context lengths. NAMOH achieves this by activ…
-
New method verifies object claims in multimodal LLMs
Researchers have developed a new training-free method called Semantic-Spatial Agreement Verification (SSAV) to address object hallucination in multimodal large language models. This technique verifies object claims by a…
-
New Relation-Orbit method enhances VLM spatial claim verification
Researchers have developed a new method called Relation-Orbit for improving the accuracy of vision-language models (VLMs) in verifying claims that involve spatial relationships, such as left-right distinctions. This tec…
-
New privacy defense prunes visual tokens for LLMs
Researchers have developed QPriv-VL, a novel framework designed to enhance privacy in Vision-Language Models (VLMs) used in sensitive applications like Federated Learning. This system intelligently prunes visual tokens …