LLMs and new frameworks boost GPU kernel optimization
ByPulseAugur Editorial·[15 sources]·
Researchers are exploring novel ways to optimize GPUkernel performance for large language models. One approach uses language models as surrogates to predict kernel performance, significantly increasing the number of candidates considered within a limited budget. Another method, STOF, accelerates sparse Transformers by optimizing multi-head attention and fusing downstream operators. Additionally, a new framework called KLineage learns optimization skills from expert kernels to guide LLMs, while Xe-Forge automates kernel optimization for Intel GPUs using a multi-stage pipeline. Finally, FastKernels addresses the gap between benchmark performance and real-world deployment by creating production-aligned benchmarks and inference frameworks.
AI
IMPACT
New LLM-based techniques and benchmarks promise to accelerate GPUkernel optimization, potentially leading to faster AI model inference and deployment.
RANK_REASON
Multiple research papers introducing new methods and benchmarks for GPU kernel optimization using LLMs and other techniques.
arXiv:2605.31464v1 Announce Type: cross Abstract: GPU kernels are the workhorse of modern deep learning, and optimizing them (via evolutionary search or coding agents) usually requires repeated measurement on target hardware. While these measurements provide the ground-truth sign…
GPU kernels are the workhorse of modern deep learning, and optimizing them (via evolutionary search or coding agents) usually requires repeated measurement on target hardware. While these measurements provide the ground-truth signal necessary for kernel search, they are costly, b…
arXiv:2506.06095v5 Announce Type: replace Abstract: Large language models (LLMs) are popular around the world due to their powerful understanding capabilities. As the core component of LLMs, accelerating Transformer through parallelization has gradually become a hot research topi…
arXiv:2605.28213v1 Announce Type: new Abstract: LLM-based agents are increasingly used to generate GPU kernels, but they often know what optimizations to try without knowing when those optimizations are sound. We introduce KLineage, which learns this missing "when" knowledge from…
arXiv cs.AI
TIER_1English(EN)·Marcin Spoczynski, Daniel Fleischer, Moshe Berchansky, Gabriela Ben-Melech Stan, Shira Guskin, Weilin Xu, Adam Siemieniuk, Alexander Heinecke·
arXiv:2605.26118v1 Announce Type: cross Abstract: Porting deep learning algorithms to new hardware accelerators requires developers to repeatedly apply the same low-level optimizations -- quantization, memory access coalescing, tile size tuning, and architecture-specific workarou…
arXiv:2605.23215v1 Announce Type: cross Abstract: LLM-based agents for GPU kernel generation are advancing rapidly, yet their progress is fundamentally constrained by the benchmarks they optimize against. Existing benchmarks are poorly aligned with production inference frameworks…
LLM-based agents for GPU kernel generation are advancing rapidly, yet their progress is fundamentally constrained by the benchmarks they optimize against. Existing benchmarks are poorly aligned with production inference frameworks: they evaluate kernels on a single GPU with synth…
LLM-based agents for GPU kernel generation are advancing rapidly, yet their progress is fundamentally constrained by the benchmarks they optimize against. Existing benchmarks are poorly aligned with production inference frameworks: they evaluate kernels on a single GPU with synth…
FastKernels addresses the gap between benchmark evaluation and production performance for LLM kernel agents by providing a representative set of architectures and a production-grade inference framework that aligns evaluation with real-world deployment.
<p>UC Berkeley's UCCL team releases mKernel, fusing intra-node NVLink, inter-node RDMA, and dense compute into a single persistent CUDA kernel.</p> <p>The post <a href="https://www.marktechpost.com/2026/05/29/meet-mkernel-a-multi-gpu-multi-node-fused-kernel-library-for-gpu-driven…
Medium — MLOps tag
TIER_1English(EN)·Parv Agarwal·
UC Berkeley's UCCL team releases mKernel, a fused CUDA kernel library that merges GPU communication and compute into a single persistent kernel. Communication can consume over 40% of AI training time - this approach aims to eliminate that bottleneck. https://www. marktechpost.com…
UC Berkeley's UCCL team releases mKernel, a fused CUDA kernel library that merges GPU communication and compute into a single persistent kernel. Communication can consume over 40% of AI training time - this approach aims to eliminate that bottleneck. https://www. marktechpost.com…
<!-- SC_OFF --><div class="md"><p>This is what I worked on today. With local LLM of course. So if I didn't write the code, did I really work on it? Who cares. It was my idea and I simply asked it to implement it. I basically downloaded /main/ branch, which is totally broken for W…
<!-- SC_OFF --><div class="md"><p>Profiling PyTorch training has an interesting measurement problem: the more you measure, the more you can change the behavior of the run itself.</p> <p>A simple example is <code>torch.cuda.synchronize()</code>. It gives cleaner timing boundaries,…
