Imperfect Visual Verification for Code Edition : A Case Study on TikZ
A new paper explores the effectiveness of iterative refinement in LLM-based code editing, particularly for tasks involving visual outputs like TikZ diagrams. The study investigates how imperfect verifiers, which are necessary when formal evaluation is impossible, impact the refinement process. Findings indicate that even unreliable verifiers can achieve moderate accuracy in confirming instruction application, with feedback improving customization success rates, especially for less capable models. AI
IMPACT Investigates challenges in LLM code customization for visual outputs, suggesting iterative refinement with imperfect feedback can still yield moderate success.