New FrenchNews-7 Benchmark Evaluates LLMs on News Classification

Researchers have introduced FrenchNews-7, a new benchmark for classifying French news articles by editorial desk. This benchmark combines a large corpus of French news from multiple publishers with a seven-class taxonomy derived from URLs. A fine-tuned CamemBERT model achieved the best performance, outperforming headline-only inputs and zero-shot large language models like GPT-OSS-120B, Mistral Small 3.2, and Llama-3.3-70B Instruct. The study found that while some categories like 'Sport' and 'International' are classified reliably, 'Economie' and 'Societe' present challenges, indicating limitations in classifier headroom rather than just editorial boundary ambiguity. AI

IMPACT Establishes a new benchmark for evaluating LLM performance on French news classification, highlighting areas where current models struggle with nuanced editorial boundaries.

RANK_REASON The cluster describes a new academic paper introducing a benchmark dataset and evaluation of models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• arXiv
• CamemBERT
• CamemBERT-base
• FrenchNews-7
• GPT-OSS 120B
• Hugging Face
• Llama 3.3 70B Instruct
• Mistral Small 3.2

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →