A homelabber has significantly upgraded their AI build by adding two RTX 3090 GPUs to a system featuring Optane memory, enabling them to run the 2.8-trillion-parameter Kimi K3 model. This configuration achieves a generation speed of 3.11 tokens per second and 16.8 tokens per second for prompt evaluation. By moving nearly half of the model's shards (414 GB out of 865 GB) into a RAM drive, performance saw a 41% boost in generation speed and a 26% increase in prompt evaluation. AI
IMPACT Demonstrates advanced homelab techniques for running large models locally, potentially inspiring other enthusiasts.
RANK_REASON This is a user-driven hardware modification and performance tuning of an existing AI model, not a release from a frontier lab or significant industry event.
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →