SGLang
PulseAugur coverage of SGLang — every cluster mentioning SGLang across labs, papers, and developer communities, ranked by signal.
- 2026-10-02 product_launch SGLang released version 0.5.21, adding support for DeepSeek-V4.1 Flash. source
- 2026-09-05 product_launch SGLang released version 0.5.19, featuring a large number of contributions. source
- 2026-08-22 product_launch SGLang released version v0.5.18, featuring 710 pull requests from 212 contributors. source
- 2026-08-22 product_launch SGLang released version 0.5.18. source
- 2026-07-25 product_launch SGLang released version 0.5.16, including the new DSpark speculative decoding algorithm. source
- 2026-07-23 controversy A critical vulnerability, CVE-2026-14890, was disclosed in SGLang, enabling unauthenticated remote code execution. source
- 2026-06-20 product_launch SGLang and MUSA merge backends, enabling native GPU support for China's open-source AI ecosystem. source
- 2026-01-09 product_launch SGLang released version 0.3.1 of its model gateway, featuring performance and memory improvements. source
28 day(s) with sentiment data
-
NVIDIA NIXL and LMCache slash LLM latency with Disaggregated Prefill-Decode
A new technique called Disaggregated Prefill-Decode, utilizing NVIDIA's NIXL and LMCache, significantly improves LLM inference performance, particularly for long-context tasks. This method separates the compute-intensiv…
-
NVIDIA Dynamo releases lag behind latest engines, snapshots riskier
A recent analysis of NVIDIA Dynamo releases reveals that stable versions often lag behind the latest available engine updates, with pinned vLLM and SGLang versions being a median of 30 days old on release day. The study…
-
SGLang vs. vLLM: Prefix Caching Performance Showdown
SGLang and vLLM are two systems designed to optimize LLM inference, particularly through prefix caching. SGLang utilizes RadixAttention, a CPU-based prefix cache that stores prompt and answer KV caches in a radix tree, …
-
Researchers find limited KV cache reduction potential in standard transformer layers
Researchers investigated the potential for KV cache reduction in standard transformer models, inspired by LoopVQ's success with looped transformers. They found that while some redundancy exists between adjacent layers i…
-
Backboard.io engine beats vLLM, SGLang on Qwen3.8-27B inference
Backboard.io has developed a new inference engine that outperforms vLLM and SGLang in terms of throughput and latency when running the Qwen3.8-27B model on a single GPU. The engine achieved superior results across vario…
-
Backboard.io's new inference engine outperforms vLLM and SGLang
Backboard.io has developed a new inference engine that significantly outperforms existing solutions like vLLM and SGLang in terms of throughput and latency. Across various models including GLM-5.3, Qwen3.6-35B, and Qwen…
-
DSpine enhances LLM inference with parallel draft token generation
Researchers have developed DSpine, a novel method for speculative decoding in large language models that generates draft tokens in parallel. This approach enhances efficiency by passing predicted predecessor features be…
-
XingChen-AGI releases 29B MoE model with 256K context on Ascend NPU
XingChen-AGI has released Xing4.0-29B-A4B, a 29 billion parameter language model with 4 billion active parameters per token. This model boasts a 256K context length, expandable to 512K, and is optimized for complex engi…
-
Baidu releases ERNIE 5.5 Flash Base, requiring significant hardware and restricted use
Baidu has released ERNIE 5.5 Flash Base, a 280B parameter open-weight model, on Hugging Face. This model is a raw pre-training checkpoint and does not include chat capabilities out-of-the-box, requiring fine-tuning for …
-
KVBuffer enhances linear attention serving for long-context AI models
Researchers have introduced KVBuffer, a novel serving mechanism designed to enhance the efficiency of linear attention models, particularly for long-context inference. This IO-aware approach buffers recent keys and valu…
-
Cloudflare Workers AI leaks customer prompts due to KV cache bug
A bug in Cloudflare's Workers AI service, specifically within the KV cache mechanism for the GLM 5.3 model, inadvertently exposed parts of previous customer prompts to subsequent users. This memory bookkeeping error occ…
-
Alibaba Cloud integrates Moonshot AI's Kimi K3.1 Pro on new SuperPoD
Alibaba Cloud has announced that its Lingjun Zhenwu M890 SuperPoD now offers day-zero support for Moonshot AI's Kimi K3.1 Pro model. This integration leverages vLLM and SGLang for efficient operation. The system also in…
-
New method accelerates triangular inversion for linear transformers
A new paper published on arXiv introduces an improved method for triangular matrix inversion, a critical operation in Delta-Rule Linear Transformers. This technique is crucial for efficient long-context models like Qwen…
-
LLM Serving Engines: vLLM vs. SGLang Performance Under Scrutiny
A new research paper published on arXiv evaluates the performance of LLM serving engines, specifically vLLM and SGLang. The study found that default configurations can lead to significant performance disparities, with v…
-
New SEED method optimizes hybrid LLM memory and compute efficiency
A new research paper introduces SEED, a method designed to optimize the memory and compute efficiency of hybrid language models. SEED addresses the inefficiencies in how these models handle memory by making the prefill …
-
Huawei Ascend enables day-zero support for Moonshot AI's Kimi K3.1 Pro
Huawei's Ascend team has achieved day-zero inference support for Moonshot AI's Kimi K3.1 Pro, a 2.8-trillion-parameter model. This integration allows the Kimi K3.1 Pro model to run on Huawei's Atlas A3 and Ascend 950 Su…
-
LoopVQ Quantization Method Halves KV Cache Memory for Looped Transformers
Researchers have developed LoopVQ, a novel quantization method for the key-value (KV) cache in looped transformers. This architecture, which reuses layers across multiple steps, faces memory bottlenecks due to its KV ca…
-
LoopVQ method slashes KV cache memory for looped transformers
Researchers have developed LoopVQ, a novel vector quantization method designed to significantly reduce the memory footprint of the key-value (KV) cache in looped transformers. This technique exploits redundancy across l…
-
Ant Lingbo open-sources robot AI brain, gains global traction
Ant Group's embodied AI company, Ant Lingbo, showcased its fully open-source generalist AI brain for robots at the 2026 World Internet Conference Wuzhen Summit. The LingBot series models have achieved over 2.2 million d…
-
Quantization of open LLMs offers speedups but lacks consistent accuracy reporting
A recent analysis of mid-size open LLMs reveals that quantization techniques, while promising speedups, often come with a trade-off in accuracy that is not consistently reported across all models and benchmarks. The stu…