Descriptive versus Regulatory Uncertainty in Bounded Predictive Systems
A new paper distinguishes between descriptive uncertainty, which merely describes output distributions, and regulatory uncertainty, which actively influences a system's policy and drives adaptation. The research demonstrates that current transformer architectures are limited to descriptive uncertainty during inference. This limitation is explained through Landauer's principle, suggesting that for uncertainty to be regulatory, epistemic error must incur a real energy cost, which is not the case in decoupled systems. Empirical tests on language models of varying sizes showed that token-level entropy remained statistically invariant across different tasks, indicating a decoupling from task accuracy and a scale-invariant limitation. AI
IMPACT This research suggests a fundamental limitation in current transformer architectures regarding adaptive behavior, potentially requiring new approaches for genuine epistemic grounding.