Researchers have developed a new method for creating function vectors (FVs) to steer Large Language Models (LLMs) during in-context learning. The study explores variations in FV definitions, focusing on attention head selection and steering techniques. By employing gradient-based attributions with Layer-wise Relevance Propagation (LRP) for head selection and a distributed approach for steering, the method significantly enhances both efficiency and accuracy in guiding LLMs. AI
IMPACT Introduces a more efficient and accurate method for controlling LLM behavior, potentially improving performance on various downstream tasks.
RANK_REASON The cluster contains an academic paper detailing a new method for LLM steering.
- Large Language Models (LLMs)
- Layer-wise Relevance Propagation (LRP)
- Function Vectors
- Large Language Models
- Layer-wise Relevance Propagation
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →
