Sparse few-shot language model for Bengali achieves 90% sparsity

Researchers have developed BnBERT-iPET, a novel approach to sparse few-shot language modeling specifically for Bengali. This method utilizes lottery ticket pruning to achieve 90% sparsity, significantly reducing computational requirements and memory footprint. The pruned model demonstrates performance comparable to larger, state-of-the-art models like Bangla Electra and Indic-BERT on Bengali downstream tasks. AI

IMPACT Enables more efficient development and deployment of NLP models for resource-constrained languages.

RANK_REASON The cluster contains an academic paper detailing a new model and methodology for NLP. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• Bengali
• Bangla Electra
• BERT
• BnBERT-iPET
• Indic-BERT
• lottery ticket hypothesis
• XLM-RoBERTa

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →