Researchers have developed OraclePhys, a novel framework for fine-tuning large language models (LLMs) on structural mechanics problems. This framework includes a benchmark with an automated scoring system, a dataset of seven answer forms, and a controlled training study. The study revealed that the answer format, rather than the length of the label, dictates what LLMs learn during fine-tuning. Specifically, ranking objectives install an out-of-distribution forward model, while scalar objectives provide only partial capability. The trained 8B model, the first LLM capable of understanding spatial structural response, achieved state-of-the-art performance on the task, outperforming existing LLMs in zero- and few-shot settings. AI
IMPACT This research offers a new methodology for fine-tuning LLMs on specialized scientific domains, potentially improving their capabilities in fields like engineering and physics.
RANK_REASON The cluster contains an academic paper detailing a new framework and findings for LLM fine-tuning. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
