Attention Calibration for Position-Fair Dense Information Retrieval
Researchers have developed a method to mitigate positional bias in dense retrieval models, a common issue where retrieval effectiveness decreases as relevant information appears later in a passage. This new technique, adapted from inference-time attention calibration, can be applied without retraining the models. By interpolating between original and calibrated attention distributions, partial calibration was found to frequently outperform full calibration across various embedding models and datasets. AI
IMPACT This technique offers a way to improve retrieval system performance without costly retraining.