Brief

The developer of llama-launcher, a GUI for creating llama-server commands, has released version 1.3. This update introduces a new feature that utilizes Bayesian optimization, specifically Tree-Structured Parzen estimation via the optuna framework, to automatically tune model parameters. Initial testing with Gemma 12B MTP models has shown up to a 15% improvement in speeds without manual intervention. AI

Two developers have released open-source tools to simplify the use of llama.cpp, a popular framework for running large language models locally. One tool, llama-launcher, offers a point-and-click graphical interface for managing llama-server flags, making it accessible for beginners. The other, start-llama, provides command-line customization for multiple llama-server binaries with per-model overrides, aiming for easy, single-step execution. AI