On-device NER models evaluated for accuracy, cost, and reliability

A new study published on arXiv evaluates nine named-entity recognition (NER) systems for on-device deployment, considering accuracy, cost, reliability, and confidence. The research compares classical taggers, bidirectional-encoder specialists, and generative large language models (LLMs) across various parameter sizes and datasets. Findings indicate that while larger LLMs are competitive in accuracy, smaller encoder models offer significant advantages in deployability due to their reduced size and faster latency, though generative models can produce a notable percentage of invalid output. AI

IMPACT Provides insights into selecting and evaluating NER models for efficient on-device deployment, balancing accuracy with resource constraints.

RANK_REASON The cluster contains a research paper detailing a study on named-entity recognition models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• DeepSeek-R1-1.5B
• DeepSeek-R1:8b
• Gliner
• Hugging Face
• Qwen3 0.6B
• Qwen3 1.7B
• Qwen3-4B-Instruct
• SpaCy
• Vinay Kumar Chaganti

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →