Brief

A recent benchmark evaluated six large language models on their ability to extract structured data, specifically JSON, from customer support emails. The analysis found that Anthropic's Claude Haiku 4.5 offered the best value, achieving high accuracy at a significantly lower cost compared to more powerful models. While Gemini 2.5 Flash was fast and inexpensive, it struggled with accuracy, particularly in hallucinating data. The study suggests using Haiku for most extraction tasks, Sonnet for more complex reasoning, and avoiding more expensive frontier models for simple data extraction. AI

A software development team experienced a silent regression when migrating from OpenAI's GPT-4o to GPT-4.1, as a subtle change in the model's output format broke their customer support ticket summarization tool. The issue, where a field name changed from 'urgency' to 'urgency_level', bypassed standard testing because the JSON remained valid and unit tests focused on the prompt string, not its output. To prevent such 'silent regressions' in the future, the article recommends implementing a dedicated testing framework like PromptFork, which can compare model outputs against a baseline and flag even minor format or reasoning drifts. AI