vLLM has released version 0.34.0, which includes the removal of Model Runner V1. This update also adjusts the default block size to 64 and GPU memory utilization to 1.0. Additionally, the new version adds support for Mistral Large 4 and several other models. AI
IMPACT Improves efficiency and expands model compatibility for LLM inference.
RANK_REASON This is a software release for a tool that facilitates LLM inference, not a frontier model release.
Read on Mastodon — mastodon.social →
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →