Auroc
PulseAugur coverage of Auroc — every cluster mentioning Auroc across labs, papers, and developer communities, ranked by signal.
12 day(s) with sentiment data
-
New research highlights pitfalls in LLM collusion monitoring
A new research paper published on arXiv identifies significant pitfalls in evaluating multi-agent collusion monitors for large language models (LLMs). The study reveals that current probes often detect whether an agent …
-
New theory reveals flaws in common OOD detection methods
Researchers have identified an "attraction-ranking mismatch" in out-of-distribution (OOD) detection methods. This mismatch occurs when features are attracted towards in-distribution regions, but this attraction does not…
-
New PLUS method enhances near-OOD detection in machine learning models
Researchers have developed a new method called PLUS (Per-class Low-dimensional sUbspace Signal) to improve out-of-distribution (OOD) detection in machine learning models, particularly for near-OOD scenarios. Unlike exis…
-
New AI protocol struggles to predict future MS lesions beyond population averages
Researchers have developed FORESEE-MS, a new evaluation protocol aimed at predicting future multiple sclerosis (MS) lesions from baseline FLAIR MRI scans. The study tested five deep learning models against a population-…
-
New method improves LLM reliability detection by transferring domain signatures
Researchers have developed a new method to improve the reliability detection of large language models (LLMs) by adaptively transferring representational signatures across related domains. This approach addresses the lim…
-
New DIGOM framework enhances multimodal healthcare outcome prediction
Researchers have developed DIGOM, a novel graph-based framework designed to improve outcome prediction in multimodal healthcare data. DIGOM explicitly models three types of interactions: within-patient cross-modal, cros…
-
New scaling laws predict safety probe performance for language models
Researchers have developed scaling laws to predict the performance of safety probes used to monitor language models. By analyzing six new safety datasets and training probes on 20 base models across five families, they …
-
AI agents vulnerable to malicious Markdown files; System 1 models show mixed results in medicine
A new study published on Hugging Face evaluates the performance of System 1 models in medical contexts, finding they excel at knowledge recall and text understanding but struggle with image interpretation and clinical j…
-
New AI method links radar heart sounds to cardiac dysfunction phenotypes
Researchers have developed a new method using radar heart sounds to analyze cardiac mechanical activity and pair it with echocardiographic data. This approach, detailed in a new paper, aims to better understand ventricu…
-
New protocol offers rigorous equivalence testing for ML experiments
A new research paper proposes a protocol for rigorously testing the equivalence of machine learning training methods. The protocol, which includes power theory and a two-level bootstrap, aims to certify practical equiva…
-
SHAP attribution methods show weak causal faithfulness in language models
Researchers have evaluated the effectiveness of SHAP as an attribution method for sparse autoencoder (SAE) features in language models. Their study, using sentiment classification on the SST-2 benchmark with GPT-2 Small…
-
New 'Derivation Gap' Found in LLM Tool-Call Argument Verification
Researchers have identified a "derivation gap" in how LLM agents' tool-call judges evaluate proposed argument values. These judges are less likely to accept a correct value if its source is not explicitly present in the…
-
New research stresses reliability of connectome-constrained AI models
A new paper, "FlyHack: Stress-Testing Mechanistic Claims in a Connectome-Constrained Visual Model," explores the reliability of conclusions drawn from connectome-constrained neural networks. The research, which tested a…
-
New CoPoE framework tackles missing data in Alzheimer's diagnosis
Researchers have developed CoPoE, a novel framework for multimodal Alzheimer's disease diagnosis that effectively handles missing data. CoPoE maps various types of clinical evidence into a structured latent space repres…
-
AI models reveal distinct internal dynamics in epilepsy patients despite similar predictive accuracy
Researchers have developed a method to analyze the internal dynamics learned by AI models, even when their predictive accuracy is similar. A CNN-Transformer model, trained on EEG data from the Temple University EEG Corp…
-
LLM routing systems advance with token-level efficiency and cost-saving strategies · 9 sources tracked
New research and practical implementations are exploring advanced methods for routing requests to Large Language Models (LLMs) to optimize cost and quality. Papers like "TokenRouter" propose efficient serving systems fo…
-
Audit questions exercise-specific joint selection benefits in AI classification
A new audit of exercise-specific joint selection in skeleton-based correctness classification reveals that while it can improve accuracy, the gains are often marginal and depend heavily on evaluation methods. The study …
-
AI model predicts AML mutations from flow cytometry data
Researchers have developed an interpretable multi-instance learning model that can predict key molecular alterations in acute myeloid leukemia (AML) from routine flow cytometry data. This approach, which models patient …
-
Small LLMs like Qwen2.5 and Llama 3.2 exhibit significant capitulation to user pushback
A new research paper investigates the tendency of small language models, specifically Qwen2.5-1.5B and Llama-3.2-1B, to abandon correct answers when challenged by users. The study found that these models frequently swit…
-
New research questions VLM confidence calibration, proposes new evaluation metric
A new research paper published on arXiv, "The Mirage of Calibrated Confidence," reveals that vision-language models (VLMs) often report high confidence in their answers regardless of the reasoning process they followed.…