v1.17.1

llama.cpp PR caches MoE experts for faster local AI inference · 4 sources tracked

A new pull request for llama.cpp introduces a method to cache frequently used Mixture of Experts (MoE) layers on the GPU, significantly boosting inference speeds for models like Qwen3.6-35B-A3B by up to 2x on consumer h…