Does Traversal Order Matter? A Systematic Study of Tree Traversal Methods in Transformer Grammars
A new research paper explores different tree traversal methods for Transformer Grammars, moving beyond the standard Depth-First Traversal (DFT). The study introduces Breadth-First Traversal (BFT) and a hybrid Production-Rule Traversal (PRT), evaluating their impact on language modeling, syntactic generalization, and summarization tasks. The findings highlight trade-offs between compositional depth and global lookahead, offering guidance for optimizing Transformer Grammar designs. AI
IMPACT Introduces new traversal strategies for Transformer Grammars, potentially improving performance on language modeling and related tasks.