GEMM
PulseAugur coverage of GEMM — every cluster mentioning GEMM across labs, papers, and developer communities, ranked by signal.
6 day(s) with sentiment data
-
AI agents co-design hardware-software infrastructure for faster model performance
Researchers have developed a new framework called Formulaic that uses AI agents to co-design AI infrastructure, integrating hardware and software for improved performance. This approach allows AI agents to propose desig…
-
OpenAI's Sora generates realistic video from text, raising creative and ethical questions
OpenAI has introduced Sora, a text-to-video generative AI model capable of creating up to one-minute videos from text prompts. While Sora demonstrates a sophisticated understanding of the physical world and can produce …
-
BiLoCo method cuts LLM memory use with binary low-rank corrections
Researchers have developed BiLoCo, a novel method for post-training quantization (PTQ) that significantly reduces the memory footprint of large language models (LLMs) while maintaining accuracy. This technique stores lo…
-
New Waypoint model optimizes GEMM for faster LLM inference on CPUs
Researchers have developed Waypoint, an analytical model designed to optimize the performance of General Matrix Multiplication (GEMM) on CPUs. By considering architectural factors like memory hierarchy and multi-threadi…
-
llama.cpp adds Q2_K quantization support for OpenCL on Adreno GPUs
The llama.cpp project has released version b11873, which introduces initial support for the Q2_K quantization format within its OpenCL backend. This update includes new kernel implementations for General Matrix Vector M…
-
FIGFLEX accelerator co-designed for low-bit LLM quantization
Researchers have developed FIGFLEX, a novel mixed-precision accelerator designed to optimize low-bit quantization for large language models (LLMs). This hardware is co-designed with quantization techniques to efficientl…
-
Ant Group, vLLM open-source Humming for quantized LLM inference
Ant Group and the vLLM community have jointly released Humming, an open-source library for just-in-time compiled GEMM kernels. This library is designed to accelerate quantized LLM inference, supporting a range of intege…
-
llama.cpp releases pre-versions with JSON fix and OpenCL enhancements
The llama.cpp project has released two pre-release versions, b11871 and b11789. Version b11871 introduces a fix for JSON nesting depth limits to prevent server crashes and includes support for various operating systems …
-
llama.cpp b11763 adds OpenCL GEMM variants and OS support
The llama.cpp project has released version b11763, introducing several new OpenCL variants for GEMM operations, particularly for small batch sizes. These updates include fixes for out-of-bounds reads and the removal of …
-
New TME Model Term Accounts for Residue Deconstruction Cost
A new paper introduces a missing fourth term, the residue deconstruction cost, to the Tensor-Memory Equilibrium (TME) model. This term accounts for the per-input deconstruction cost required before matrix multiplication…
-
New SSR method accelerates ternary LLM inference
Researchers have developed Sparse Segment Reduction (SSR), a new method for accelerating the inference of ternary Large Language Models (LLMs). This approach optimizes matrix multiplication for ternary weights, which ar…
-
New T-CCL library boosts multi-GPU AI model performance with TMA
Researchers have developed T-CCL, a new collective communication library designed for efficient multi-GPU execution in large transformer models. T-CCL leverages the Tensor Memory Accelerator (TMA) to offload data moveme…
-
Space Filling Curves simplify communication-avoiding matrix multiplication
A new research paper introduces Space Filling Curves (SFC) as a method to simplify and optimize Communication-Avoiding General Matrix Multiplication (CA-GEMM) for high-performance computing and deep learning. This appro…
-
New tools and research tackle GPU optimization for AI workloads
Several research papers and a new open-source tool address challenges in optimizing AI workloads on GPUs. COMPASS-ABS aims to reduce fragmentation in shared GPU clusters for deep learning training, improving resource ut…
-
AI alignment advocates call for regulation of GEMM and GEMV operations
A call has been made to regulate GEMM and GEMV operations, which are fundamental to AI alignment and model execution. The proposal suggests that entities utilizing BLAS (Basic Linear Algebra Subprograms) without proper …
-
Together AI optimizes ThunderKittens for NVIDIA Vera Rubin Blackwell GPUs
Together AI has gained access to NVIDIA's Vera Rubin NVL72 platform, which is based on the Blackwell architecture. Their team has updated their ThunderKittens software to leverage new features of the Vera Rubin chip, sp…
-
New im2win convolution method boosts GPU performance and memory efficiency
Researchers have developed an enhanced version of the im2win convolution method, designed for greater memory efficiency and performance on GPUs. This updated method supports full precision on CUDA cores and half precisi…
-
Hand-written PTX kernels show significant speedups for INT8/INT4 GEMM on NVIDIA L4 GPUs
A new research paper explores the performance benefits of using hand-written PTX (Parallel Thread Execution) kernels for GEMM (General Matrix Multiply) operations on NVIDIA L4 GPUs, compared to the standard WMMA (Warp M…
-
InstantSfM offers GPU-native structure-from-motion for deep learning era
Researchers have developed InstantSfM, a novel GPU-native structure-from-motion system designed to integrate seamlessly with deep learning pipelines. This system addresses the limitations of traditional CPU-centric SfM …
-
MiniCPM-V 4.6 multimodal assistant runs on 2011 GPU
Researchers have successfully deployed the MiniCPM-V 4.6 multimodal assistant on a 2011 NVIDIA Tesla C2075 GPU, which has 6GB of memory. This involved creating an all-GPU inference engine optimized for the older Fermi a…