Research: RL and SFT Differently Teach Transformers Boolean Functions

A new research paper explores how transformers learn sparse Boolean functions, comparing the distinct mechanisms of Reinforcement Learning (RL) with process rewards and Supervised Fine-Tuning (SFT). The study identifies conditions under which transformers can provably learn these functions, demonstrating this for k-PARITY, k-AND, and k-OR functions. Key findings reveal that RL learns the entire reasoning chain simultaneously, while SFT learns it step-by-step, offering insights into the underlying learning dynamics of these fine-tuning approaches. AI

IMPACT Provides theoretical insights into how different fine-tuning methods impact transformer learning capabilities for specific reasoning tasks.

RANK_REASON This is a research paper published on arXiv detailing theoretical findings about transformer learning dynamics. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv stat.ML →

• Bochen Lyu
• Boolean functions
• k-PARITY
• Reinforcement Learning
• Supervised Fine-Tuning
• Transformers

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →