New benchmark 'TalkFa' released for Farsi dialogue generation and understanding

Researchers have introduced TalkFa, a new benchmark designed to evaluate Farsi language dialogue systems. The benchmark includes three datasets: Wiki-FADIAL for knowledge-grounded generation, DAILYDIALOG-FA for dialogue acts and emotions, and PLAYDIAL-FA for theatrical dialogues and sentiment analysis. Experiments using various LLAMA and Mistral models demonstrated that LoRA fine-tuning significantly improves dialogue generation, and specific models like FABERT and LORA-MISTRAL-7B showed strong performance on classification tasks. The benchmark was validated by native speakers and external assessments, revealing that automatic metrics often overestimate dialogue quality. AI

IMPACT Provides a crucial resource for advancing Farsi NLP, enabling better evaluation and development of dialogue systems for a large speaker population.

RANK_REASON The cluster describes a new academic benchmark for Farsi dialogue generation and understanding, including datasets and experimental results. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• DAILYDIALOG-FA
• FABERT
• Farsi
• GPT-4.1
• LLAMA
• LoRA
• LORA-MISTRAL-7B
• MISTRAL-24B
• PLAYDIAL-FA
• TalkFa
• Wiki-FADIAL

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →