New dataset and models improve German NLG quality evaluation

Researchers have developed TextQ-German, a new dataset and suite of models for evaluating the quality of German natural language generation (NLG) from a human-centered perspective. Traditional automatic metrics are insufficient for assessing perceived quality, so this work focuses on Quality of Experience (QoE). The dataset includes human ratings for tasks like summarization and machine translation, and the developed models, particularly hybrid approaches combining transformers and linguistic features, show strong performance in predicting these human QoE scores. AI

IMPACT This research provides a new resource for evaluating German language generation models, potentially leading to more human-aligned AI outputs.

RANK_REASON The item describes a new dataset and evaluation models for natural language generation, which falls under research. [lever_c_demoted from research: ic=1 ai=1.0]

Read on Hugging Face Daily Papers →

• German
• Hugging Face
• large-language models
• natural language generation
• Quality of Experience
• TextQ-German

• paper
• product

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →