The Nanbeige4.3-33B, an open-source agent coding model, can be run on a single 24 GB GPU by carefully managing model weights and the "thinking mode." Running the dense 33 billion parameter model requires fitting its weights entirely within the GPU's VRAM to maintain speed, as offloading to slower system RAM significantly degrades performance. Developers have found that using a 3-bit quantized weight file (around 14.7 GB) on a 16 GB GPU achieves optimal speeds of over 13 tokens per second, while a 2-bit file is too imprecise for coding tasks. A bug related to the chat template can cause responses to appear in the "reasoning" field instead of the "content" field, which can be mitigated by disabling thinking mode or adjusting chat template arguments. AI
IMPACT Enables running larger agent models on consumer-grade hardware, potentially lowering the barrier to entry for self-hosting and experimentation.
RANK_REASON The item details how to run a specific existing model on consumer hardware, including performance tuning and bug workarounds.
- Boss Zhipin
- Groot-19B
- K. E. Bartowski
- llama.cpp
- Nanbeige4.3-33B
- Nanbeige Team
- .openai
- RTX 2000 Ada
- RTX 4090
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
