ChaosNLI
PulseAugur coverage of ChaosNLI — every cluster mentioning ChaosNLI across labs, papers, and developer communities, ranked by signal.
5 day(s) with sentiment data
-
Soft-label training enhances classifier accuracy by using full annotation distributions
Researchers have proposed a new training method called soft-label training for supervised classifiers, which utilizes the full distribution of annotator labels rather than a single majority vote. This approach is partic…
-
NLI label variation study fails to replicate prior findings
A preregistered replication study on the Stanford Natural Language Inference corpus, MultiNLI, and ChaosNLI datasets has failed to confirm prior findings regarding human label variation. The original research suggested …
-
NLI label variation study fails to replicate prior findings on monotonicity
A preregistered replication study aimed to verify a previously observed boundary in human label variation (HLV) within natural language inference (NLI) tasks. The original study suggested that hypotheses with non-upward…
-
Formal semantic structure explains minimal human label variation in NLI tasks
A new research paper explores the extent to which formal semantic structure explains human label variation in natural language inference (NLI) tasks. The study analyzed items from the SNLI and MNLI corpora, finding that…
-
New research finds temperature scaling fails on soft labels
A new research paper challenges the effectiveness of temperature scaling for model calibration, particularly when dealing with soft or distributional human labels. The study found that temperature scaling, which assumes…
-
Apple ML Research: Annotation needs vary by evaluation metric
Apple Machine Learning Research has published a paper detailing a method called Metric-Dependent Annotation Saturation. This approach suggests that the number of annotators required to capture meaningful signal from lab…
-
Annotation needs for AI models vary by evaluation metric, study finds
A new research paper explores how the number of annotators needed to effectively train AI models depends on the specific evaluation metric used. The study, focusing on Natural Language Inference (NLI) models, found that…