A new auditing protocol called TERRA (Test-time Error Response and Realizability Audit) has been introduced to evaluate AI models. TERRA is an auditing protocol, not a new inference algorithm, and it decomposes intervention outcomes into applicability, target-state realization, endpoint translation, harm, and resource/inference cost. Experiments with executable financial reasoning revealed that context expansion can have varied effects on different configurations, sufficient evidence does not guarantee correctness due to translation differences, and diagnostic headroom does not always ensure practical gains. AI
IMPACT Introduces a new framework for evaluating AI model reliability and safety, particularly in complex reasoning tasks.
RANK_REASON The item describes a new auditing protocol for AI models presented in a research paper. [lever_c_demoted from research: ic=1 ai=1.0]
Read on Hugging Face Daily Papers →
- capability-conditioned test-time inference
- diagnostic headroom
- endpoint translation
- executable financial reasoning
- Harm
- local consistency
- program-consistency
- rendering
- resource/inference cost
- Retrieval
- State realization with exogenous variables - A test on blast furnace data
- state-transition audit
- target-state realization
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →