PulseAugur
EN
LIVE 00:37:57
한국어(KO) 111GB 모델 파일이 램 78.8GB로 도는 이유: POCKET GGUF를 비트 수·KV 캐시·토큰당 읽는 가중치로 분해한 램 예산표

LLM RAM Usage: Beyond File Size with POCKET GGUF Analysis

By PulseAugur Editorial · [1 sources] ·

This article delves into the complexities of running large language models locally, explaining why simply looking at file size is insufficient for RAM allocation. It highlights that KV cache can grow beyond the model file size with longer contexts, and that Mixture of Experts (MoE) models don't necessarily load their entire file into RAM. The analysis breaks down RAM usage by examining bits per weight, KV cache size, and the actual weights read per token for various POCKET GGUF models, offering a more accurate method for estimating RAM needs. AI

IMPACT Provides a more accurate method for estimating RAM requirements for running LLMs locally, enabling better hardware planning.

RANK_REASON Technical analysis of LLM memory usage and performance characteristics. [lever_c_demoted from research: ic=1 ai=1.0]

Read on dev.to — LLM tag →

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →

LLM RAM Usage: Beyond File Size with POCKET GGUF Analysis

How we ranked this

Signal score
49 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Tool
Technical analysis of LLM memory usage and performance characteristics. [lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration
Single-source cluster
Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics
infra, model release
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
Breaking (< 6h)
Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

Full methodology in our editorial standards.

COVERAGE [1]

  1. dev.to — LLM tag TIER_1 한국어(KO) · Vidraft_lab ·

    Why a 111GB Model File Runs on 78.8GB RAM: RAM Budget Table Decomposing POCKET GGUF by Bit Count, KV Cache, and Weights Read Per Token

    <h2> TL;DR </h2> <p>로컬에서 모델을 돌릴 때 "파일이 몇 GB냐"만 보고 램을 정하면 두 방향으로 어긋납니다. 문맥이 길어지면 KV 캐시가 파일 밖에서 자라고, 반대로 전문가 혼합(MoE) 모델은 파일 전체를 램에 올리지 않아도 돕니다. 저희가 허깅페이스에 올린 POCKET GGUF 파일 8개의 크기와 파라미터 수, 그리고 26B·35B·180B 빌드의 헤더와 텐서 목록을 직접 읽어 계산한 결과는 이렇습니다.</p> <ul> <li>파일 크기를 전체 파라미터 수로 나누면 가중치 하…

RELATED ENTITIES

RELATED TOPICS