A new benchmark, MALFORMED-300, evaluates how well parsers can extract JSON from malformed LLM outputs, particularly when JSON is wrapped in other formats like XML, HTML, or SQL. The json-repair tool achieved a perfect score of 25/25 in this specific category, outperforming jsonshim (17/25) and standard libraries (0/25). However, across the entire 300-case benchmark, jsonshim still leads with an overall score of 94.0%, while json-repair follows at 88.3%. AI
IMPACT Improves LLM output parsing reliability, potentially reducing errors in applications that consume LLM-generated structured data.
RANK_REASON The item details the performance of a specific tool, json-repair, on a benchmark for parsing malformed LLM outputs.
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
