LLM alignment for code generation: pretrained vs. fine-tuned models

Researchers have explored the effectiveness of Large Language Model (LLM) alignment techniques for code generation tasks, investigating whether alignment should start with a pretrained or a fine-tuned LLM. The study utilized Direct Preference Optimization (DPO) and BoNBoN, two reward-free alignment methods, on five state-of-the-art LLMs. Results indicated that aligning pretrained models led to greater improvements in the aligned versions compared to their pretrained counterparts, although the pretrained models were generally less accurate. Conversely, aligning fine-tuned models yielded smaller performance gains or even degradation. AI

IMPACT Investigates optimal strategies for aligning LLMs for code generation, potentially improving the quality and maintainability of AI-generated code.

RANK_REASON The cluster contains an academic paper detailing empirical study results on LLM alignment for code generation. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• Direct Preference Optimization
• EvalPerf
• EvoEval
• HumanEval+
• Large Language Model
• MBPP+
• SelfCodeAlign

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →