FlashInfer
PulseAugur coverage of FlashInfer — every cluster mentioning FlashInfer across labs, papers, and developer communities, ranked by signal.
5 day(s) with sentiment data
-
NVIDIA boosts MoE inference speed on DGX Spark with 3 optimizations
NVIDIA has detailed three methods to enhance the inference speed of Mixture-of-Experts (MoE) models on their DGX Spark desktop system. By implementing a new MoE kernel backend, an FP8 output layer, and integrated specul…
-
Evolving Agent Teams framework tackles AI coding optimization challenges
A new multi-agent framework called Evolving Agent Teams has been developed to overcome local optima in AI-driven software engineering, particularly for GPU kernel optimization. This framework utilizes a supervisor to di…
-
Stratified Sparse Attention boosts LLM long-context inference speed and accuracy
Researchers have developed Stratified Sparse Attention (SSA), a new framework designed to improve the efficiency and accuracy of large language model inference over long contexts. SSA combines deterministic retrieval of…
-
New research optimizes sparse attention for LLMs with QUILT and PageWeaver
Two new research papers, QUILT and PageWeaver, propose novel methods to optimize sparse attention mechanisms in large language models. QUILT introduces a workload-aware execution strategy that reuses shared KV entries a…
-
Cleave compiler optimizes tensor programs for large models
Researchers have developed Cleave, a new machine learning compiler designed to optimize tensor programs for large models. Cleave employs a strategy of symbolic decoupling, first performing superoptimization on a graph w…
-
Prime Intellect launches Prime Inference serving platform for open models
Prime Intellect has launched Prime Inference, a new serving platform designed for frontier open-source AI models. The platform offers both serverless endpoints for variable demand and reserved capacity for sustained wor…
-
vLLM releases 0.30.0rc1 with FlashInfer autotuning fix
vLLM has released version 0.30.0rc1, which includes a bugfix to isolate supplemental FlashInfer BF16 autotuning. This release was co-authored by jiahanc and OpenAI Codex.
-
NVIDIA DGX Spark GB10: vLLM installation and performance guide
A technical guide details how to install and run vLLM on NVIDIA's DGX Spark (GB10) hardware without relying on containerized environments. The guide highlights specific Python version requirements and potential installa…
-
Understanding GPU Kernels: The Key to Efficient AI Inference
The article explains the concept of a 'kernel' in the context of GPU computation for AI models, defining it as a single function executed by thousands of threads in parallel. It highlights that the primary challenge in …
-
New frameworks optimize GPU kernels for deep learning and HPC · 3 sources tracked
Three new research papers introduce advanced frameworks for optimizing GPU kernels, crucial for deep learning and high-performance computing. HIERA focuses on workload-aware planning across different implementation spac…
-
Kimi K3 LLM self-hosting costs $89.52/hr, offers 1M context
Kimi K3, a large language model developed by Moonshot AI, has provided a performance report from its own operational environment. Running on eight NVIDIA B300 SXM6 GPUs with a total of 2.2 TB of VRAM, the model boasts a…
-
New WAM-Diff2 framework boosts autonomous driving VLA model efficiency
Researchers have developed WAM-Diff2, a new framework designed to improve the efficiency of Vision-Language-Action (VLA) models for autonomous driving. This framework uses a hierarchical distillation strategy to convert…
-
New research explores LLM efficiency and reasoning improvements
Several research papers explore methods to enhance the efficiency and reliability of large language models (LLMs). Hugging Face's LFM2.5-DSpark demonstrates up to 3.2x faster inference speeds by using speculative decodi…
-
LLMs advance GPU kernel generation and inference optimization · 4 sources tracked
Researchers are developing advanced methods for optimizing GPU kernel generation using large language models (LLMs). One approach, presented at MLSys 2026, uses a harness-centered system to constrain, validate, and prof…
-
DeepSeek V4 Flash with DSpark shows significant speed gains over EAGLE
A user on Reddit shared their experience deploying the DeepSeek V4 Flash model using DSpark via SGLang on an HGX-H200 system. They compared DSpark's performance against EAGLE, finding DSpark to be significantly faster, …
-
PersistentKV optimizes LLM serving on commodity GPUs with new scheduling techniques
A new paper introduces PersistentKV, a system designed to optimize the serving of large language models (LLMs) with long contexts on commodity GPUs. PersistentKV employs page-aware decode scheduling and a native block-t…
-
MiniMax M3 integrates with NVIDIA hardware, vLLM, and Inferact
SemiAnalysis reported on the successful integration of MiniMax AI's M3 model with NVIDIA's hardware, specifically highlighting the vLLM project and Inferact's EAGLE3 spec decode. This collaboration focuses on enabling d…
-
New framework speeds up LLM inference on NVIDIA H20 GPUs
Researchers have developed FlashMLA-ETAP, a new framework designed to significantly speed up the inference of large language models on NVIDIA H20 GPUs. The framework introduces an Efficient Transpose Attention Pipeline …
-
New Qrita Algorithm Boosts LLM Sampling Efficiency
Researchers have developed Qrita, a novel algorithm designed to enhance the efficiency of Top-k and Top-p sampling in large language models. By employing Gaussian-based sigma-truncation and a quaternary pivot search, Qr…
-
Fireworks AI flags numerical drift in LLM training vs. serving
Fireworks AI has identified critical numerical parity bugs that can arise when training and serving large language models, particularly Mixture-of-Experts (MoE) architectures. These discrepancies, stemming from the non-…