Local LLM user questions RAM usage with Qwen 27B model

A user is experiencing unexpected RAM usage while running a large language model locally, despite expecting the context cache to be primarily handled by VRAM. They are using Qwen 27B with llama.cpp and a memory extension, noting that system RAM increases significantly as the context cache fills. The user is seeking clarification on whether RAM is supposed to be used for the cache and what process is causing this increased RAM consumption during inference. AI

RANK_REASON User question about local LLM resource management, not a new release or significant industry event.

Read on r/LocalLLaMA →

• 3090
• Hermes-memory
• llama.cpp
• Qwen 27B

• infra
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →