A new study benchmarks six frontier large language models (LLMs) on their ability to extract structured crash data from police narratives, comparing their performance to official crash coding. The research utilized 5,587 fatal-crash narratives from Arkansas and found that while GPT-5.5 High showed the highest agreement among the LLMs, a simple always-majority baseline achieved better raw agreement. Performance varied significantly across different crash attributes, with non-motorist relation and crash manner being easier to code than light condition or roadway surface condition. The study suggests that LLM deployment for crash coding should be evaluated on an attribute-specific basis against transparent baselines and human review. AI
IMPACT Provides a benchmark for LLM performance in extracting structured data from unstructured text, relevant for applications in transportation safety and data analysis.
RANK_REASON Academic paper presenting benchmark results for LLMs on a specific task. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
