Exposing the Unsaid: Visualizing Hidden LLM Bias through Stochastic Path Aggregation
Researchers have developed TreeTracer, a visual analytics tool designed to uncover hidden biases in Large Language Models (LLMs). Unlike traditional methods that inspect single outputs or use static metrics, TreeTracer aggregates hundreds of stochastic generations into a hierarchical structure. This allows for a more comprehensive comparison of semantic contexts and aids in detecting representational harms such as pronoun suppression and conversational marginalization. Case studies comparing GPT-2 XL with Apertus models demonstrated TreeTracer's effectiveness in exposing these biases. AI
IMPACT Provides a novel method for identifying and mitigating biases in LLMs, potentially leading to fairer and more reliable AI systems.