The ExLlamaV3 project has released version 1.0.0, marking a significant performance upgrade after over a year of development. This release introduces a new attention kernel with advanced quantization and caching, improved GEMM/GEMV performance on Ampere hardware, and new kernels for INT8 and MoE operations. ExLlamaV3 also now supports tensor parallelism for a wider range of models, including Gemma4, and has removed dependencies on flash-attention-2 and xformers. AI
IMPACT Improves inference performance and model compatibility for local LLM deployments.
RANK_REASON This is a release of a specific inference engine/library, not a frontier model release.
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
