Limited Marginal Benefit of Reasoning-Heavy LLM Deployment in ESG Narrative Scoring: A 4-Model Consensus Study on Japanese Listed Firms
A new study published on arXiv suggests that using advanced, reasoning-heavy large language models (LLMs) for ESG narrative scoring offers limited marginal benefit compared to less computationally intensive models. The research, which analyzed data from ten Japanese listed firms, found that the cost of employing these frontier models was approximately 5.6 times higher than using a consensus of three reasoning-off models, with only minor differences in scoring outcomes. The findings imply that for span-based ESG narrative scoring, the increased operational cost of reasoning-heavy LLMs does not yield materially improved results. AI
IMPACT The study indicates that for specific applications like ESG scoring, simpler and cheaper LLMs may suffice, potentially influencing deployment strategies and cost-benefit analyses in AI adoption.
