Anything Goes? A Crosslinguistic Study of (Im)possible Language Learning in LMs
A new research paper explores whether language models (LMs) can offer insights into human language learning by training them on typologically unattested languages. The study, which focused on 12 languages and used GPT-2 small, found that while the model could largely distinguish between attested and impossible languages, it did not achieve perfect separation. Furthermore, GPT-2 small's performance on generalization tests indicated some human-like inductive biases, though these were less pronounced than in human learners. AI
IMPACT Suggests LMs may exhibit some human-like inductive biases in language acquisition, though less pronounced than in humans.