FlashAttention-2
PulseAugur coverage of FlashAttention-2 — every cluster mentioning FlashAttention-2 across labs, papers, and developer communities, ranked by signal.
2 day(s) with sentiment data
-
New framework enables exact out-of-memory recovery for LLM attention
Researchers have developed Stream-CQSA, a novel framework designed to overcome out-of-memory (OOM) limitations in long-context large language models. This method utilizes CQS decomposition to recursively partition atten…
-
New methods drastically shrink LLM size and boost inference speed
Researchers have developed two novel methods to significantly reduce the size and computational cost of large language models (LLMs) without substantial performance loss. Squeeze10-LLM employs a staged mixed-precision q…
-
Google Cloud releases Open Knowledge Format for AI agent memory
Google Cloud has introduced the Open Knowledge Format (OKF), a specification designed to help AI agents retain and access crucial information. OKF uses plain markdown files with YAML frontmatter to store knowledge, such…
-
Run Qwen 3.8-27B locally with DeepSeek Harness and Unsloth
A technical guide details how to run the Qwen 3.8-27B code model locally on a Windows 11 machine with an RTX 3090 graphics card. The setup leverages DeepSeek Harness for agent orchestration and Unsloth Engine for optimi…
-
FlashAttention-2 & 3 Tackle GPU Memory Limits for LLMs
This article delves into the technical advancements of FlashAttention-2 and FlashAttention-3, explaining how they overcome the limitations of GPU memory bandwidth. It details the use of algorithmic tiling and asynchrono…
-
GRACE system accelerates real-time ad retrieval with generative recommenders
A new research paper introduces GRACE, a system designed to accelerate generative recommenders for real-time ad retrieval. GRACE addresses challenges in eligibility and compute by implementing Generative Target Matching…
-
Fine-tuning VRAM bottleneck identified: Loss tensor consumes majority of memory
A technical analysis reveals that a significant portion of VRAM during LoRA fine-tuning is consumed by a temporary cross-entropy loss tensor, rather than the model itself. This tensor, which exists only briefly to produ…
-
Eddy-VL 1.9B: Compressed multimodal model for edge deployment
Researchers have developed Eddy-VL 1.9B, a compressed multimodal embedding model designed for edge deployment in environments without cloud access. Built upon Qwen3-VL-Embedding-2B, Eddy-VL utilizes structural pruning a…
-
Kernelized Linear Attention Boosts Model Capacity and Throughput
Researchers have developed Kernelized Linear Attention Activations (KATA), a new framework designed to overcome the capacity limitations of linear attention models. KATA utilizes symmetric cones and rank-one positive se…
-
FlashAttention-3/4 optimizations ineffective on consumer RTX GPUs
An exploration into FlashAttention-3 and FlashAttention-4 optimizations revealed that these advanced techniques are not applicable to consumer-grade RTX GPUs. The research found that while FlashAttention-2 achieves pari…
-
Picotron framework enables LLM training on older GPUs
A developer has created Picotron, an LLM training framework designed to run on older GPUs without crashing. This framework eliminates mandatory GPU-specific dependencies, allowing it to function on any GPU supporting Py…
-
New Causal-rCM recipe accelerates autoregressive video diffusion
Researchers have introduced Causal-rCM, a novel open recipe for autoregressive video diffusion distillation. This framework unifies teacher-forcing and self-forcing paradigms to enhance streaming video generation and in…
-
Subquadratic AI unveils SubQ 1.1 Small with 12M token context
Subquadratic AI has released its new model, SubQ 1.1 Small, which utilizes Smart Sparse Attention to achieve near-perfect long-context retrieval up to 12 million tokens. This model significantly reduces computational re…
-
SubQ unveils SubQ 1.1 Small with 12M-token context and sparse attention
SubQ has released its SubQ 1.1 Small model, featuring a new Subquadratic Sparse Attention (SSA) architecture designed to overcome the quadratic scaling limitations of traditional attention mechanisms. This new architect…
-
ByteDance releases Bernini open-source video generation framework
ByteDance has released Bernini, an open-source framework for video generation and editing. The system combines a multimodal large language model for semantic planning with a DiT-based renderer. Bernini reportedly achiev…
-
Stanford's ThunderKittens DSL optimizes AI kernel performance
A new article details ThunderKittens, a compact domain-specific language (DSL) developed at Stanford's Hazy Research Lab for creating high-performance AI kernels. The DSL aims to strike a balance between research produc…
-
Sigmoid attention improves biological foundation models with faster, stable training
Researchers have developed a new attention mechanism called Sigmoid Attention, which offers significant improvements for training biological foundation models. This novel approach leads to better learned representations…
-
Google AI optimizes cloud computing with LAVA, Together AI expands GPU cloud, and Modal streamlines AI/ML deployment
Google DeepMind researchers have developed LAVA, a new AI-driven scheduling algorithm designed to optimize resource allocation in cloud data centers. LAVA continuously re-predicts virtual machine (VM) lifetimes, adaptin…