Attention Expansion: Enhancing Keyphrase Extraction from Long Documents with Attention-Augmented Contextualized Embeddings
Researchers have developed an "attention expansion" mechanism to improve keyphrase extraction from long documents. This method augments pre-trained language model (PLM) representations with information from surrounding text using word embeddings, effectively broadening the model's contextual scope without needing full-document attention or costly LLM inference. Evaluations across various PLM backbones and datasets show consistent performance gains, establishing attention expansion as an efficient strategy for long-document KPE. AI
IMPACT Enhances efficiency and effectiveness of information retrieval from lengthy texts, potentially improving research and content analysis.