New metric reveals how language models process metaphor

Researchers have developed a new metric called conditional scale entropy (CSE) to analyze how decoder-only language models process metaphors. CSE measures the breadth of computational engagement across different frequency scales within a transformer's layers. Studies using CSE revealed that metaphorical tokens consistently activate a wider range of computational scales compared to literal tokens in models ranging from 124 million to 20 billion parameters, including architectures like GPT-2, LLaMA-2, and GPT-oss. AI

IMPACT Introduces a novel metric for understanding metaphorical processing in LLMs, potentially aiding in the development of more nuanced language understanding capabilities.

RANK_REASON The cluster contains an academic paper detailing a new method for analyzing language model behavior. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• conditional scale entropy (CSE)
• decoder-only language models
• GPT-2
• GPT-oss 20B
• LLaMA-2 7B

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →