New CREG Benchmark Measures Evidence-Grounded Belief Updating in Robotics

Researchers have introduced the Connected Robotics Evidence Graph (CREG) Benchmark, a new evaluation protocol designed to measure evidence-grounded shared belief updating in connected robotics systems. The benchmark comprises 350 scenarios across seven regimes, providing a detailed trajectory of how systems attribute evidence, update beliefs, and handle contradictions. Initial evaluations of four open-weight instruction models and six fusion baselines revealed that entity grounding significantly impacts performance, and that the semantic relations within evidence are crucial for accuracy, even outperforming graph serialization in some cases. The study also highlighted that belief accuracy can be non-monotonic throughout the evidence sequence, a nuance often missed by endpoint-only evaluations. AI

IMPACT This benchmark could lead to more robust and reliable AI systems in robotics by enabling detailed evaluation of belief updating and evidence handling.

RANK_REASON The cluster describes a new benchmark and evaluation protocol for robotics research published on arXiv.

Read on Hugging Face Daily Papers →

• alphaXiv
• arXiv
• CatalyzeX
• Connected Robotics Evidence Graph
• DagsHub
• Gotit.pub
• Hugging Face
• ScienceCast
• Crégy-lès-Meaux
• Holm-corrected
• Internet of Things

• paper
• product

AI-generated summary · Google Gemini · from 2 sources. How we write summaries →