MAPLE project claims 26B LLM runs 6x faster on 6GB GPU

A new open-source project called MAPLE claims to have optimized a Gemma 4 26B-A4B large language model to run at 45.4 tokens per second on a consumer-grade RTX 4050 laptop GPU with 6 GB of VRAM. This is a significant speed increase compared to the original model, which reportedly achieved only 6.9 tokens per second under different conditions. However, the claim requires careful scrutiny, as the MAPLE model is pruned and uses all 30 layers on the GPU, unlike the original model which had layers offloaded. Independent verification is needed to assess the model's actual file size, runtime compatibility, generation speed, and task quality against the original. AI

IMPACT This development could enable more users to run capable LLMs locally on less powerful hardware, potentially increasing accessibility and use cases for AI applications.

RANK_REASON The item discusses a specific optimization and benchmark for running a large language model on consumer hardware, which is a tool-related development rather than a core model release or significant industry event.

Read on dev.to — LLM tag →

• 45.4 tokens per second
• 6.9 tokens per second
• Gemma 4 - 26B-A4B
• GGUF
• Python
• RTX 4050

• product
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →