MALFORMED-300
PulseAugur coverage of MALFORMED-300 — every cluster mentioning MALFORMED-300 across labs, papers, and developer communities, ranked by signal.
-
New JSON recovery tool refuses to invent data for LLM errors
A new tool called jsonshim-mcp has been released to address the challenge of recovering malformed JSON outputs from language models. Unlike other libraries, jsonshim-mcp invents no values for unrecoverable JSON and refu…
-
json-repair excels at extracting LLM-wrapped JSON, outperforming jsonshim
A new benchmark, MALFORMED-300, evaluates how well parsers can extract JSON from malformed LLM outputs, particularly when JSON is wrapped in other formats like XML, HTML, or SQL. The json-repair tool achieved a perfect …
-
LLM JSON parsers struggle with truncated output; Toolkit Labs releases benchmark
Toolkit Labs has released findings from a benchmark of JSON parsers designed to handle malformed output from large language models. The study focused on 25 cases of truncated JSON, where a model's output is cut off mid-…
-
Python JSON parsers benchmarked on malformed LLM outputs
A benchmark test was conducted on seven Python JSON parsers to evaluate their performance on malformed outputs from large language models (LLMs). The test suite, MALFORMED-300, included 300 cases of malformed JSON. The …
-
Developer creates 300-case suite to test LLM JSON parsing robustness
A developer has created a 300-case conformance suite, named MALFORMED-300, to test the robustness of Large Language Model (LLM) parsers when handling malformed JSON output. The suite categorizes errors into twelve types…