Frozen Foundation-Model Embeddings Discard Small-Lesion Signal in Chest Radiography: Implications for Pre-Deployment Evaluation
A new research paper investigates how frozen foundation-model embeddings in vision transformers (ViTs) impact the detection of small lesions in chest X-rays. The study found that standard aggregation methods like classification tokens and patch means discard crucial small-lesion signals. However, when embeddings are restricted to specific regions of interest, the signal is recoverable, achieving near-perfect accuracy. AI
IMPACT Standard aggregation methods in AI models suppress critical signals in medical imaging; restricting analysis to regions of interest can recover this lost information.