A series of posts explores the memory requirements for running various large language models (LLMs) locally on Mac computers with different unified memory configurations. The analysis details how model size, parameter count, and quantization levels impact whether a model can fit within the available GPU memory, particularly for configurations ranging from 16GB to 128GB. Specific models like gpt-oss, Gemma 4, LFM2.5, Qwen, and Nemotron are discussed in relation to their memory footprint on Macs equipped with MLX. AI
IMPACT Provides practical guidance for users looking to run LLMs locally on consumer hardware, detailing memory requirements for various models and Mac configurations.
RANK_REASON The cluster provides practical guidance on running existing LLMs on consumer hardware, rather than announcing new models or research.
Read on Mastodon — mastodon.social →
- 128GB
- 30b Parameter Model
- 32GB
- 48gb Ram
- 64Gb/s PAM4 and 160Gb/s 16QAM modulation reception using a low-voltage Si-Ge waveguide-integrated APD
- bromine-70
- Gemma 4
- gpt-oss
- graphics processing unit
- LFM2.5
- Mac
- macOS Sequoia
- MLX
- Nemotron
- Qwen
AI-generated summary · Google Gemini · from 3 sources. How we write summaries →