InferenceX
PulseAugur coverage of InferenceX — every cluster mentioning InferenceX across labs, papers, and developer communities, ranked by signal.
- developed jalapeño 95%
- developed Together Compiler 95%
- used by jalapeño 70%
- instance of Together Compiler 70%
- competes with Kimi k3 70%
- competes with DeepSeek V4.1 Flash 70%
- competes with MiniMax M3.1 Flash 70%
- competes with GLM 5.3 60%
- used by DeepSeek V4.1 Flash 50%
- used by Agent Extensibility Protocol 50%
- used by GLM 5.3 50%
- used by MiniMax M3.1 Flash 50%
6 day(s) with sentiment data
-
Sarvam launches India-hosted inference platform with 8 open models
Sarvam has launched the Sarvam Token Factory, an inference platform hosted in India that provides access to eight open-weight models via a single OpenAI-compatible API. The service utilizes NVIDIA B200 clusters and aims…
-
Together AI unveils compiler for continuous AI inference optimization
Researchers at Together AI have developed the Together Compiler (TKCC), a novel system designed for continuous optimization of AI inference workloads. TKCC addresses the challenge of rapidly evolving AI models and diver…
-
Together AI Compiler boosts inference throughput up to 2.7x on NVIDIA hardware
Together AI has released benchmarks showcasing significant performance improvements with their compiler. When compared against optimized InferenceX configurations on NVIDIA B200 and B300 hardware, Together Compiler demo…
-
New runtime compilation boosts AI model throughput by 2.6x
A new runtime compilation technique has been developed that significantly boosts throughput for AI models. This method achieved up to a 2.6x increase in transactions per second (TPS) compared to standard InferenceX setu…
-
Together AI unveils Together Compiler for continual AI inference optimization
Together AI has introduced the Together Compiler (TKCC), a new system designed for continual compilation in AI inference. This innovation aims to automatically enhance performance and ensure accuracy for AI models. The …
-
Together Compiler optimizes AI inference, while Flide accelerates decision models · 7 sources tracked
Together AI has introduced the Together Compiler, an agent-driven system designed to continuously optimize AI inference for production traffic. This system aims to address the dynamic nature of AI workloads by adapting …
-
NVIDIA SGLang Rubin GPU shows 4.23x performance per dollar advantage
SemiAnalysis reports that NVIDIA's SGLang inference engine, utilizing the Rubin NVL72 GPU, demonstrates a significant performance and cost advantage over the GB300 NVL72. The SGLang engine, integrated with the Qwen3.8 F…
-
SemiAnalysis discusses GPU economics, InferenceX, and new hardware
SemiAnalysis discussed the economics of GPUs, featuring the InferenceX team's insights on Engram memory offloading and AgentX's profit calculator. The conversation also touched upon TPU v7, a comparison between Vera Rub…
-
GLM5.3 Sparse Attention Mechanism Impacts HBM Memory Usage
SemiAnalysis has detailed how GLM5.3's sparse attention mechanism impacts High Bandwidth Memory (HBM) usage. The analysis covers techniques like KV Cache Offloading and HiSparse, which are crucial for optimizing perform…
-
OpenAI's custom Jalapeño AI chip prioritizes internal use, hints at future rollout
OpenAI has developed a custom AI inference ASIC named Jalapeño, primarily for its internal use to meet growing compute demands. While the chip is designed to be programmable and capable of running various models beyond …
-
Vera Rubin NVL72 Agentic Inference System Claims 67x Performance Boost
SemiAnalysis reports on Vera Rubin NVL72, an agentic inference system that claims to offer a 67x improvement in performance per dollar. The system also boasts double the annual profit per gigawatt and a model where incr…
-
TPUv7 introduces SparseCore for 12% better throughput, outperforming Blackwell Ultra
The TPUv7 features a specialized hardware unit called SparseCore, designed to manage data movement by grouping expert tokens. This optimization, when used with the TensorCore for matrix multiplications, boosts throughpu…
-
Google TPUv7 Ironwood outperforms NVIDIA Blackwell Ultra on inference tasks
SemiAnalysis reports that Google's TPUv7 Ironwood hardware offers a 50% improvement in performance per dollar compared to NVIDIA's Blackwell Ultra for inference tasks. This advantage is realized through the new TorchTPU…
-
InferenceX externalizes TPU stack, challenging CUDA dominance
SemiAnalysis reports that InferenceX is making significant strides in TPU inference externalization, potentially offering up to 50% better performance per dollar. The company is actively developing its TPU stack, with n…
-
OpenAI unveils custom Jalapeño ASIC for inference workloads
OpenAI has developed an in-house inference ASIC named Jalapeño, designed in collaboration with Broadcom. This custom chip aims to provide the optimal compute platform for OpenAI's inference workloads, focusing on perfor…
-
OpenAI's custom 'jalapeño' chip benchmarks show it beating NVIDIA hardware
OpenAI has released benchmark results for its custom inference chip, codenamed "jalapeño," which it developed in collaboration with Broadcom. The chip reportedly outperforms NVIDIA's GB300 and GB200 systems in throughpu…
-
OpenAI shares performance data for custom Jalapeño inference chip
OpenAI has released performance data for its custom inference chip, codenamed Jalapeño. The chip reportedly achieved higher peak throughput per kilowatt and lower token latency than existing commercial systems when test…
-
OpenAI's Jalapeño chip shows superior inference performance over Nvidia
OpenAI has revealed initial performance data for its custom-designed "Jalapeño" inference chip, showcasing significant improvements in speed and power efficiency. Benchmarks indicate that Jalapeño outperforms competitor…
-
SemiAnalysis revises Morgan Stanley's GenAI ROIC calculations
SemiAnalysis is critiquing a Morgan Stanley report on generative AI return on invested capital (ROIC), suggesting adjustments to capital expenditure calculations. While Morgan Stanley's report is generally bullish on Ge…
-
SemiAnalysis discusses DeepSeek V4, Huawei Ascend NPU, and LLM framework competition
SemiAnalysis has released an episode discussing the DeepSeek V4 model and the performance of Huawei's Ascend NPU. The episode, titled "Ep. 017 - DeepSeek V4 and Huawei Ascend NPU Performance (InferenceX)", features insi…