Distributional Alignment as a Criterion for Designing Task Vectors in In-Context Learning
Researchers have introduced a new metric, $d_{\text{NTP}}$, to evaluate the effectiveness of task vectors in large language models by measuring the discrepancy in next-token probabilities between task vector-based and in-context learning (ICL) inference. This metric serves as a performance proxy, correlating negatively with downstream accuracy. Based on this, they developed the Linear Task Vector (LTV) method, which improves average accuracy by 9.2% and reduces inference latency across various benchmarks and LLMs. LTV also demonstrates transferability, enhancing smaller models' performance by 6.4% when using task vectors from larger models. AI
IMPACT Enhances LLM efficiency and accuracy in task adaptation, potentially reducing inference costs and improving performance transfer across model scales.