PulseAugur
EN
LIVE 05:14:05

LM Studio update causes RTX 3090 performance drop; CUDA runtime switch fixes issue

By PulseAugur Editorial · [1 sources] ·

A user experienced a significant performance drop in their local LLM setup after updating LM Studio, with their RTX 3090's processing speed decreasing from 60-70 tokens/sec to 12-18 tokens/sec. The issue was traced to only 54 layers being offloaded to the GPU, despite settings indicating maximum offload. The problem was resolved by switching the runtime from CUDA12 to either CUDA or Vulkan, with CUDA offering slightly better performance. The user noted that CUDA12 appeared to be forcing CPU offload. AI

IMPACT A software update for local LLM deployment tools can unexpectedly degrade performance, requiring users to adjust runtime settings.

RANK_REASON User-reported issue with a specific software update affecting hardware performance.

Read on Mastodon — fosstodon.org →

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →

LM Studio update causes RTX 3090 performance drop; CUDA runtime switch fixes issue

COVERAGE [1]

  1. Mastodon — fosstodon.org TIER_1 English(EN) · [email protected] ·

    Today’s local LLM adventure: after updating LM Studio, my RTX 3090 dropped from 60-70 tk/s to 12-18. Logs showed only 54 layers offloaded to GPU, despite max GP

    Today’s local LLM adventure: after updating LM Studio, my RTX 3090 dropped from 60-70 tk/s to 12-18. Logs showed only 54 layers offloaded to GPU, despite max GPU offload and GPU-only settings. The rest was on sys CPU + RAM. The annoying fix was switching from CUDA12 to CUDA or Vu…

RELATED ENTITIES

RELATED TOPICS