Scheduled sampling's effectiveness poorly predicted by exposure gap metric

A new analysis of scheduled sampling in sequence prediction models reveals that the "exposure gap" metric, often cited to address exposure bias, provides minimal predictive power regarding the effectiveness of scheduled sampling. The research suggests that the model's performance under standard teacher forcing is a much stronger indicator of whether scheduled sampling will be beneficial. The study also highlights issues with the fixed point of the procedure and the accuracy of measuring model error, particularly with recurrent neural networks. AI

IMPACT This research suggests that current evaluation methods for sequence models may be misleading, potentially impacting how researchers develop and assess new models.

RANK_REASON The item details a new analysis and findings regarding a specific machine learning technique (scheduled sampling) and its evaluation metrics. [lever_c_demoted from research: ic=1 ai=1.0]

Read on dev.to — LLM tag →

• exposure bias
• Exposure Gap
• recurrent neural network
• Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
• teacher forcing

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →