Triton
PulseAugur coverage of Triton — every cluster mentioning Triton across labs, papers, and developer communities, ranked by signal.
17 day(s) with sentiment data
-
vLLM 0.33.0 mandates CUDA 13, adopts Model Runner V2, and boosts speed
vLLM has released version 0.33.0, introducing significant changes that require user attention for a smooth upgrade. The release mandates CUDA 13.0, discontinuing support for older versions and necessitating driver updat…
-
New DiRotQ method enhances 4-bit quantization for Diffusion Transformers
Researchers have developed DiRotQ, a new 4-bit quantization framework for Diffusion Transformers (DiTs) that significantly reduces memory and computational costs while preserving image generation quality. The method uti…
-
NeuroBrix 0.6.2 benchmark shows Triton engine outperforming PyTorch
A recent benchmark of the NeuroBrix 0.6.2 system reveals that its Triton engine significantly outperforms the PyTorch engine across a variety of models. In tests involving 18 different models run on Tesla V100 GPUs, the…
-
NeuroBrix simplifies AI model execution and image generation on Mac
NeuroBrix is a new open-source AI tool that allows users to run language models and generate images with a single command. It utilizes the Triton engine and operates without requiring PyTorch, with kernel settings deriv…
-
NeuroBrix 0.6 released: Unified inference engine for 49 AI models
NeuroBrix has released version 0.6, marking its first stable release. This open-source project offers a unified inference engine capable of running 49 different AI models across nine families, including language, vision…
-
torch.compile Explained: Dynamo, AOT Autograd, and Inductor
This note delves into the intricacies of torch.compile, drawing insights from IBM Technology. It details how Dynamo traces Python code to capture FX graphs and record guards, while AOT Autograd prepares the graph. Induc…
-
ZYFA boosts AMD MI355X FP8 FlashAttention speed by up to 1.99x
Researchers have developed ZYFA, a new set of prefill attention kernels for AMD Instinct MI355X GPUs that leverage FP8 precision for improved speed and accuracy. ZYFA employs novel techniques, including a simplified pro…
-
Step-dLLM boosts diffusion LLM inference speed with adaptive sparse attention
Researchers have developed Step-dLLM, a novel method to improve the efficiency of diffusion large language models (dLLMs) during inference. Unlike autoregressive models, dLLMs cannot use KV caching and recompute attenti…
-
LLM research explores GPU kernel generation and distributed computing, with CPU parallelism lessons discussed
A project has added support for multi-node tensor parallelism in its distributed computing framework, enabling communication across separate processes using NCCL for CUDA backends. Separately, a research effort has fine…
-
New PACK3D attention operator improves 3D data handling in AI models
Researchers have introduced PACK3D, a novel attention operator designed to address the challenges of packing 3D volumetric data into token sequences for AI models. Traditional methods struggle because sequence order can…
-
Developer releases ApexW PyTorch optimizer for large GPU workloads
A developer has created a new PyTorch optimizer called ApexW, designed to reduce memory usage and speed up optimizer steps for large GPU workloads. Early benchmarks show ApexW can be up to 1.5x faster than PyTorch's fus…
-
MoE Decoding Optimization Needs No Calibration Data, Accelerates Inference
Researchers have developed a new method for optimizing Mixture-of-Experts (MoE) models during decoding that eliminates the need for calibration data. The proposed 'Prox' system replaces traditional threshold-based chann…
-
US military, NASA fast-track nuclear microreactors for bases and space
The U.S. military and NASA are accelerating the development and deployment of nuclear microreactors to address power needs for remote bases and deep-space exploration. Despite a recent slowdown in the broader nuclear hy…
-
Rowhammer attack amplifies LLM inference latency and cost
Researchers have developed LLVMhammer, a novel Rowhammer attack that targets the latency and cost of Large Language Model (LLM) inference on GPUs. By inducing a bit-flip in the CPU's DRAM, the attack causes the Triton J…
-
Optimized GLM-5.3-Flash for single GPU, boosting speed 1.5x
A technical blog post details optimizations for running the large GLM-5.3-Flash model on a single 96 GiB GPU, achieving up to 16.5 tokens per second. The author implemented a "hot-expert cache" that keeps frequently use…
-
i14's Laplace 3B linear attention model trained and served on AWS Trainium2
i14 has developed Laplace 3B, a linear attention model that maintains a fixed-size recurrent state, unlike traditional transformer models whose key-value cache grows with context length. This innovation allows for more …
-
Meta optimizes AI training for AMD GPUs with co-design approach
Meta has developed an end-to-end co-design approach to optimize AI training performance across diverse hardware, specifically focusing on AMD GPUs alongside their existing NVIDIA fleet and proprietary accelerators. This…
-
BitSparse: New lossless compression format slashes LLM training memory use
Researchers have developed BitSparse, a new lossless compression format designed to reduce memory usage during the training of large language models. This format specifically targets sparse ReLU and ReLU^2 activations, …
-
bhyve Production User Call Recording Available
The recording of the October 1st, 2026, bhyve Production User Call is now available, featuring discussions on various topics including attention-requiring reviews, keelOS, and the Triton hypervisor-agnostic control plan…
-
AWS details Mixture-of-Experts kernel optimizations for SageMaker AI
AWS engineers have detailed kernel optimizations for Mixture-of-Experts (MoE) models on Amazon SageMaker AI. These optimizations focus on improving performance by fusing token routing, data shuffling, and specialized ne…