A developer testing AI image generation models found that the Qwen3-VL:32B-Thinking model was the only one capable of providing varied scores across different axes, indicating it was actually measuring the images rather than rubber-stamping them. While other models like Qwen2.5VL:7B and Qwen3-VL:30B-A3B produced consistent scores, the 32B model showed a wider range of values for humor and wit. Additionally, the 32B model was effective at identifying garbled text in generated images, a task the other models failed at. However, the 32B model exhibited an undocumented bug where it returned an empty string when asked to output in JSON format. AI
IMPACT Highlights the importance of evaluating AI model outputs for true variance rather than consistent scores, and identifies specific model capabilities and limitations.
RANK_REASON The item details a comparative evaluation of different AI models on specific tasks, including scoring and text recognition. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
