CorrSteer method enhances LLM steering using correlated sparse autoencoder features

Researchers have developed CorrSteer, a novel method for steering large language models (LLMs) during generation using features extracted from Sparse Autoencoders (SAEs). This technique correlates sample correctness with SAE activations at inference time, eliminating the need for large datasets or extensive activation storage. CorrSteer demonstrates significant performance improvements on various benchmarks, including QA, bias mitigation, and reasoning tasks, with notable gains in MMLU and HarmBench. AI

影响 Introduces a more efficient method for controlling LLM behavior, potentially improving performance on specialized tasks.

排序理由 This is a research paper detailing a new method for LLM steering. [lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.CL 阅读 →

• arXiv
• CorrSteer
• Gemma-2 2B
• HarmBench
• LLaMA-3.1 8B
• LLMs
• MMLU
• Seonglae Cho
• Sparse Autoencoders

• 论文
• 模型发布

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →