Local LLM Guide Updated with Gemma 4 Speed Boosts and Diagram Tools

Thomas Bley has updated his "Run LLMs Locally" presentation with new examples and performance improvements. The update includes a demonstration of creating Mermaid diagrams within the llama.cpp UI and introduces Quantization-Aware Training (QAT) variants for Gemma 4, which reportedly achieve 50% faster token generation on local setups. Additionally, the presentation now clarifies definitions for deterministic and probabilistic results. AI

IMPACT Provides practical guidance and performance optimizations for running LLMs locally, potentially lowering barriers for developers.

RANK_REASON Update to a guide on running LLMs locally, including performance tweaks and new examples.

Read on Mastodon — mastodon.social →

• Gemma 4
• llama.cpp
• Thomas Bley

• product
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →