semantic judge

New research reveals instability in short-answer VQA benchmarks

A new paper published on arXiv highlights significant instability in short-answer visual question answering (VQA) benchmarks. The research indicates that current benchmarks often conflate the semantic correctness of a m…