LLMs enhanced for symbolic graphics programming with RL and vision encoders

Researchers have developed a new method to improve the ability of large language models (LLMs) to generate symbolic graphics programs (SGPs), specifically Scalable Vector Graphics (SVGs), from natural language descriptions. This approach utilizes reinforcement learning with verifiable rewards, incorporating a format-validity gate for renderable SVGs and a cross-modal reward mechanism that aligns text with rendered images using vision encoders like SigLIP and DINO. Applied to the Qwen-2.5-7B model, this technique significantly enhances SVG generation quality and semantic accuracy, bringing its performance in line with leading proprietary systems. The study also introduces SGP-GenBench, a benchmark for evaluating LLMs on SVG generation tasks, covering object and scene fidelity as well as compositionality. AI

IMPACT Enhances LLM capabilities in cross-modal grounding and precise visual content generation.

RANK_REASON The cluster contains an academic paper detailing a new method for LLM program synthesis. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• Dino
• Haoquan Zhang
• large-language models
• Qwen 2.5 7B
• SGP-GenBench
• SigLIP
• SVG
• Symbolic Graphics Programs

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →