Brief

Researchers have analyzed the performance of neural networks in generating Japanese past-tense verb forms, focusing on how orthographic representations influence model accuracy. Despite high overall accuracy, the models exhibited consistent errors related to specific hiragana orthographic properties, particularly gemination. The study identified seven primary failure modes, with gemination-related errors accounting for the majority of mistakes, especially in verbs requiring stem modification before the past-tense suffix. These findings highlight the importance of considering orthography-aware evaluations for understanding neural generalization in complex languages. AI

Researchers have identified that neural morphological generation systems, despite high aggregate accuracy, often fail on rare subclasses of data. A study focusing on Japanese past-tense verb inflection revealed that a tiny fraction of irregular verbs (<1% of data) caused a disproportionate number of model errors. Removing these specific irregular patterns led to greater generalization improvements than removing all irregular verbs, suggesting that not all irregularity impacts model stability equally. AI