llama-bench defaults corrected for flash attention and GPU layers

A recent build, b9437, for the llama-bench tool has corrected default settings related to flash attention and GPU layer counts. Previously, the tool hard-coded flash attention off, even on compatible hardware, and used a legacy sentinel value for GPU layers. The update now defaults flash attention to automatic activation on capable hardware (CUDA, Metal, Vulkan) and sets the GPU layer count to -1, aligning with other llama.cpp tools like llama-server and llama-cli. This change ensures that benchmarks run with the latest defaults accurately reflect flash attention usage on supported GPUs. AI

IMPACT Ensures accurate benchmarking of flash attention on compatible hardware, improving the reliability of performance metrics for llama.cpp.

RANK_REASON This is a fix to a specific tool's default settings, not a new model release or significant industry event.

Read on dev.to — LLM tag →

• b9437
• CUDA
• flash attention
• GPU
• JohannesGaessler
• llama-bench
• llama-cli
• llama-server
• Metal
• pwilkin
• qwen3-8b-q4_k_m.gguf
• Vulkan

• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →