PulseAugur
EN
LIVE 07:15:38

Together AI pivots from megakernels to new compiler for faster AI inference

Together AI has announced their new inference optimization framework, Together Compiler, which aims to improve AI model performance. Initially, the company developed "megakernels" that fused entire model forward passes into single GPU kernels, achieving high speeds but proving difficult to productionize. Through further experimentation, Together AI found that by leveraging existing tools like CUDA Graphs and Torch Inductor, they could recover most of the megakernel performance without custom kernel writing, and even surpass previous speeds with their new compiler. AI

IMPACT Together AI's new compiler aims to optimize inference performance, potentially leading to faster and more efficient AI model deployment.

RANK_REASON The cluster describes a new product/framework release from an AI company, but not a frontier model release or significant industry-wide event.

Read on X — Together (inference / OSS) →

AI-generated summary · Google Gemini · from 2 sources. How we write summaries →

Together AI pivots from megakernels to new compiler for faster AI inference

How we ranked this

Signal score
0 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Tool
The cluster describes a new product/framework release from an AI company, but not a frontier model release or significant industry-wide event.
Source corroboration
2 independent sources
Multiple independent publishers reporting the same story raises confidence that it's real and newsworthy.
Topics
infra, product
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
2 days old
Coverage has settled into its steady-state source set.

Full methodology in our editorial standards.

COVERAGE [2]

  1. X — Together (inference / OSS) TIER_1 English(EN) · togethercompute ·

    RT @dylan__lim: Last Year We Built Megakernels. This Year We Deleted Them.

    RT @dylan__lim: Last Year We Built Megakernels. This Year We Deleted Them. One year ago, we helped introduce some of the first AI megakern…

  2. Together AI blog TIER_1 English(EN) ·

    Last year we built megakernels. This year we deleted them: The road to Together Compiler

    Together Compiler uses AI agents to rewrite and fuse the GPU kernels in a model’s forward pass. It beats hand-written megakernels by 28% on B200 (2,118 vs 1,649 tokens/s on Llama 3.2 1B). This is how we got from one to the other.