The llama.cpp project released version b12498, which includes fixes for LoRA and recurrent functionalities within its server component. This update also addresses issues related to batch processing of tokens, particularly on CUDA, ensuring more consistent test results by standardizing the processing of tokens per ubatch. AI
IMPACT Improves the stability and consistency of the llama.cpp inference server, particularly for users employing LoRA adapters or recurrent computations on CUDA hardware.
RANK_REASON This is a software release for a specific tool, not a frontier model release or significant industry event.
Read on llama.cpp — Releases →
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →