Small LLM with large lookup table matches bigger model performance

A hobby research project has demonstrated that a 21 million parameter language model, augmented with a 6.4 billion parameter lookup table, can achieve performance comparable to a 114 million parameter dense model. This augmented model is capable of running with its lookup table stored on an SSD, utilizing minimal VRAM. The project developed custom Triton kernels that are compatible across various hardware, including AMD and NVIDIA GPUs, and has made the code and model publicly available. AI

IMPACT This approach could enable smaller, more efficient models to handle complex tasks, potentially reducing hardware requirements for advanced AI applications.

RANK_REASON Research project demonstrating a novel technique for augmenting small language models with external memory. [lever_c_demoted from research: ic=1 ai=1.0]

Read on r/LocalLLaMA →

• 114M dense model
• 21M model
• 6.4B-parameter lookup table
• Claude Code
• Qwen3.5-0.8B
• Radeon RX 9070

• model release
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →