Willard & Louf (2023)

LLM JSON output requires constrained decoding, not just prompting

LLM outputs can fail to adhere to requested formats like JSON, even with explicit instructions, because prompt instructions only shift probability distributions. A more robust method is constrained decoding, which enfor…