vLLM
PulseAugur coverage of vLLM — every cluster mentioning vLLM across labs, papers, and developer communities, ranked by signal.
- 2026-06-25 product_launch vLLM released version 0.24.0rc2. source
- 2026-06-24 product_launch vLLM released version 0.24.0rc1, a release candidate that includes a fix for the topk histogram build on SM75 hardware. source
- 2026-06-04 product_launch vLLM released version 0.22.1, including a fix for DeepSeek-V4 initialization compatibility. source
- 2026-05-29 product_launch vLLM merged a pull request for a new HIP W4A16 kernel, enhancing performance. source
- 2026-05-28 product_launch vLLM released version 0.22.0rc3. source
- 2026-05-28 product_launch vLLM released version 0.22.0, including a fix for multi-API server startup timeouts. source
- 2026-05-26 product_launch Nexus Labs implemented and tested vLLM's prefix caching feature, observing significant latency improvements for AI agents. source
- 2026-05-15 product_launch vLLM released version 0.21.1rc0.
30 day(s) with sentiment data
-
Liquid AI ships tiny LFM2.5-230M for on-device agent tasks
Liquid AI has released LFM2.5-230M, its smallest model to date, designed for on-device inference on edge hardware like phones and robots. This 230-million-parameter model excels at data extraction and tool use, outperfo…
-
Discourse AI simplifies LLM backend management with Jolteon proxy
Discourse AI has developed Jolteon, a Rust-based proxy designed to manage multiple vLLM backends. This tool centralizes routing, health checks, and request adaptation for various AI models, simplifying the process of as…
-
llm-d routing layer boosts Qwen 7B inference speed by 2.3x on AWS EKS
A new routing layer called llm-d has demonstrated a significant speedup for LLM inference, specifically with the Qwen2.5-7B-Instruct model on AWS EKS. By intelligently routing requests to vLLM replicas that are likely t…
-
vLLM releases GLM-5.2 for NVIDIA Blackwell; Mixture of Agents 2.0 unveiled
The vLLM project has announced the availability of GLM-5.2 in NVFP4 format, optimized for NVIDIA's Blackwell architecture. This release enables efficient deployment of the GLM-5.2 model. Separately, Teknium introduced M…
-
DeepSeek unveils V4 models with 1M token context and MoE architecture · 3 sources tracked
DeepSeek has released preview versions of its DeepSeek-V4 series, featuring two Mixture-of-Experts (MoE) language models: DeepSeek-V4-Pro and DeepSeek-V4-Flash. Both models support an impressive one million token contex…
-
GOSIM Paris: Open-source AI focuses on reasoning transparency and data science
The GOSIM Paris conference highlighted the evolving landscape of open-source AI, with a focus on transparency in AI reasoning and the continued relevance of data science. Mathematician Sir Timothy Gowers discussed the n…
-
Ornith 1.0 models explained: Dense vs MoE and format/precision details
A guide has been released to explain the terminology and concepts behind the new Ornith 1.0 models. The guide clarifies the difference between Dense and Mixture of Experts (MoE) architectures, noting that MoE models act…
-
Tools for Local AI: vLLM Deployment, Jetson Acceleration, and Mac Containers
This week's AI news focuses on tools for local AI deployments. A Hugging Face blog post details a simplified method for setting up a vLLM server with a single command, making high-performance LLM inference more accessib…
-
Hugging Face simplifies LLM deployment with one-command vLLM server on HF Jobs
Hugging Face has introduced a new feature allowing users to deploy a vLLM server on their HF Jobs infrastructure with a single command. This simplifies the process of setting up private, OpenAI-compatible endpoints for …
-
vLLM releases 0.24.0rc2 with DP Supervisor fix
vLLM has released version 0.24.0rc2, which includes a fix for a problem related to DP Supervisor. This release was tagged by Robert Shaw and incorporates changes from commit c5e3c40877c2b6d0e16d534641b39fe6744979b7.
-
New sampler-verifier system boosts small LLM coding performance
A new research paper introduces a sampler and verifier system that significantly enhances the coding performance of small language models. This approach can potentially bring a 0.5 billion parameter model up to the leve…
-
Deploy LLMs on Kubernetes with OpenAI-Compatible API via vLLM
This guide details how to deploy an LLM on Kubernetes, focusing on exposing it as an OpenAI-compatible API. It covers setting up GPU nodes, creating a Kubernetes secret for Hugging Face tokens, and using vLLM as the mod…
-
Users discuss large model performance on RTX 6000 Ada PRO GPUs
A discussion on Reddit explores the performance of large language models like GLM 5.2, Kimi 2.7, and DeepSeek V4 Pro on high-end GPU setups featuring 4x or 8x NVIDIA RTX 6000 Ada Generation PRO cards. Users are sharing …
-
Unsloth releases Qwen-AgentWorld-35B model with broad integration support
The unsloth/Qwen-AgentWorld-35B-A3B-GGUF model is now available on Hugging Face, offering users instructions for integration with various libraries and inference providers. The model can be utilized with tools such as T…
-
LiquidAI releases compact LFM2.5-230M for on-device AI tasks
LiquidAI has released LFM2.5-230M, a compact language model designed for on-device deployment. This model boasts 230 million parameters and is optimized for efficient inference on various hardware, including CPUs and ed…
-
GLM-5.2 speculative decode runs on 4x DGX GB10 cluster
A user successfully implemented GLM-5.2 with MTP speculative decoding on a 4x DGX GB10 cluster, achieving approximately 9.4 tokens/second. This involved reconstructing missing build modifications from public kernels and…
-
NVIDIA NeMo AutoModel accelerates AI model fine-tuning
NVIDIA has released NeMo AutoModel, an open library integrated with its NeMo framework, designed to significantly accelerate the fine-tuning of large Mixture-of-Experts (MoE) AI models. This new tool builds upon Hugging…
-
GLM-5.2 model speed boosted over 20x via custom hacks
A Reddit user detailed a method for significantly accelerating the GLM-5.2 large language model on a specialized GH200 system. By combining components from different repositories and patching the vLLM inference engine, …
-
vLLM performance boosted on AMD hardware with Qwen3.5
This article details how to optimize the vLLM inference engine for AMD hardware, specifically on a Lemonade Server. The author shares their experience fixing issues and achieving a threefold increase in batch throughput…
-
User reports Qwen3.6-27B struggles with vLLM, creating custom parser
A user experienced significant performance degradation and functional issues when attempting to run the Qwen3.6-27B model using vLLM, particularly when compared to llama.cpp. Despite having ample VRAM and attempting var…