Researchers have introduced the concept of Bayes-sufficient representations for supervised learning, defining them as information that allows a prediction head to implement a Bayes-optimal action rule. This framework highlights that the relevant information depends on the specific loss function used. The paper connects this to property elicitation, showing how different losses require specific information, such as the Bayes class for zero-one loss or the conditional probability for binary prediction with Brier loss. Experiments on synthetic data, neural bottlenecks, and real-world datasets like iNaturalist illustrate the distinctions between sufficiency, minimality, and retained non-essential information. AI
IMPACT Introduces a theoretical framework for understanding information relevance in supervised learning, potentially guiding future model development.
RANK_REASON This is a research paper published on arXiv detailing a new theoretical framework for representation learning. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →