PromptNCE: Pointwise Mutual Information Predictions Using Only LLMs and Contrastive Estimation Prompts
Researchers have developed PromptNCE, a novel method that enables large language models to estimate pointwise mutual information (PMI) without requiring a separate critic model. This approach frames conditional probability estimation as a contrastive task, incorporating an 'OTHER' category to improve accuracy. PromptNCE achieves strong zero-shot performance, reaching a Spearman correlation of up to 0.82 with human-derived PMI on benchmark datasets. AI
IMPACT Enables LLMs to estimate mutual information zero-shot, potentially simplifying knowledge assessment and analysis in low-data scenarios.