Surpassing Scale by Efficiency: A Compact 135M Parameter Foundational LLM Natively Adapted for the Bangla Language
Researchers have developed a new compact language model, bangla-smollm-135m, specifically designed for the Bangla language. This 135-million parameter model achieves competitive performance against larger models by employing an efficient token merging strategy. In zero-shot evaluations, it matches or surpasses models twice its size and performs comparably to 1-billion parameter models on various benchmarks. AI
IMPACT Demonstrates that highly efficient, smaller models can achieve competitive performance, potentially enabling wider deployment of LLMs in resource-constrained environments.