Fleiss' kappa
PulseAugur coverage of Fleiss' kappa — every cluster mentioning Fleiss' kappa across labs, papers, and developer communities, ranked by signal.
2 day(s) with sentiment data
-
Humans and LLMs struggle with social media sentiment analysis, study finds
A new study published on arXiv evaluates the performance of sentiment analysis tools and large language models (LLMs) on social media texts. The research found that even human annotators exhibit only fair agreement when…
-
Word sense disambiguation bottlenecked by labels, not models, study finds
A new paper on English word sense disambiguation (WSD) highlights that current frontier LLMs are so accurate that the quality of the training labels has become the primary bottleneck for benchmark performance. The resea…
-
New VFR-Audit framework enhances reliability of AI fairness audits
Researchers have introduced VFR-Audit, a new framework designed to assess the reliability of fairness audits in clinical AI applications, specifically for predicting hospital length-of-stay. This framework focuses on th…
-
New benchmark evaluates LLMs' ability to find errors in synthetic math data
Researchers have introduced MathDebugger, a new benchmark designed to evaluate the ability of large language models to detect and diagnose errors within synthetic mathematical data. The benchmark includes a dataset of c…
-
Small language models show creativity, honesty, and designed forgetting
Researchers have demonstrated that small language models, specifically those with 146 million to 3 billion parameters, can exhibit creativity, honesty, and designed forgetting. These models, built on a hyperbolic substr…
-
SynthAVE uses LLM arena for scalable e-commerce data labeling · 2 sources tracked
Researchers have developed SynthAVE, a novel system for generating and validating synthetic labels for e-commerce attribute extraction at an industrial scale. This approach addresses the prohibitive cost of human labeli…