FlashAttention-4
PulseAugur coverage of FlashAttention-4 — every cluster mentioning FlashAttention-4 across labs, papers, and developer communities, ranked by signal.
- 2026-05-22 product_launch Together AI released FlashAttention-4, an optimized algorithm for Blackwell GPUs. source
2 day(s) with sentiment data
-
KernelSeek AI agent optimizes CUDA kernels with questions, beats baselines
Researchers have developed KernelSeek, a novel question-driven AI agent designed to optimize CUDA kernels for GPU performance. Unlike existing agents that rely on extensive toolkits or fixed policies, KernelSeek uses ta…
-
Luma Labs details multimodal sequence processing innovations
Luma Labs has released details on their advancements in efficient multimodal sequence processing, particularly for packed sequences. They have developed techniques that improve performance and reduce computational overh…
-
New polynomial approximations boost LLM training speed on NVIDIA Blackwell GPUs
Researchers have developed new polynomial approximations for transcendental functions used in large language models (LLMs) to improve computational efficiency. These approximations, tested on NVIDIA Blackwell GPUs, show…
-
New research advances tabular foundation models for efficiency and robustness
Multiple research papers explore advancements in tabular foundation models (TFMs), focusing on improving their efficiency, robustness, and adaptability. Techniques such as knowledge distillation are being used to create…
-
New FlashAttention-4 method boosts FP4 performance on Blackwell hardware
Researchers have developed a new method called Direct-P to optimize FlashAttention-4 for Blackwell's 4-bit floating-point (FP4) tensor cores, addressing performance bottlenecks caused by softmax conversion and on-chip d…
-
New research explores LLM efficiency and reasoning improvements
Several research papers explore methods to enhance the efficiency and reliability of large language models (LLMs). Hugging Face's LFM2.5-DSpark demonstrates up to 3.2x faster inference speeds by using speculative decodi…
-
New X-Stage pipeline optimization boosts DiT inference speed
Researchers have identified a new pipeline stage, termed X-Stage, that can optimize communication-computation overlap during the inference of Diffusion Transformers (DiTs). This stage focuses on the period after communi…
-
Together AI launches Inkling multimodal MoE model with 1M context window
Together AI has launched Inkling, a multimodal Mixture-of-Experts (MoE) model developed by Thinking Machines Lab. This open-weight model boasts 975 billion total parameters with 41 billion active parameters, a 1 million…
-
Thinking Machines Lab releases Inkling multimodal model with controllable reasoning
Thinking Machines Lab has launched Inkling, a new multimodal model designed for efficient reasoning and versatile task handling. The model accepts text, image, and audio inputs, and features controllable inference effor…
-
Modal optimizes FlashAttention-4 for faster LLM inference
Modal has enhanced the FlashAttention-4 kernel to improve inference speed for large language models, particularly for decode-heavy workloads. Their contributions focused on adjusting parallelism strategies, such as shif…
-
Together AI releases FlashAttention-3 and -4 for faster LLM processing
Together AI has released FlashAttention-3 and FlashAttention-4, significant upgrades to their GPU-accelerated attention mechanism for large language models. FlashAttention-3, designed for Hopper GPUs, achieves up to 75%…
-
CuTeDSL emerges as new GPU kernel path for LLM inference, challenging CUTLASS
The landscape of GPU kernel engineering for LLM inference is shifting, with CuTeDSL emerging as a potential successor to C++ CuTe/CUTLASS. This evolution is highlighted by industry trends in technologies like FlashAtten…