Brief

Researchers have developed Geo-Expert, a series of large language models specifically fine-tuned for geological reasoning. These models utilize parameter-efficient fine-tuning techniques like LoRA on base models such as Qwen3 and Gemma-3. Evaluations on a new benchmark, Geo-Eval, show that even an 8B parameter Geo-Expert model can surpass larger generalist models and GPT-4o on geological tasks, with a 32B variant nearing frontier model performance. AI

Researchers have developed a new method called Training-Trajectory-Aware Token Selection (T3S) to improve the efficiency of distilling knowledge from large language models. This technique addresses a common issue where performance metrics can drop during distillation, even as the loss decreases. T3S works by reconstructing the training objective at the token level, which helps clear the optimization path for tokens that are still learning. The method has shown consistent gains in various settings, with T3S-trained models achieving state-of-the-art performance among models of similar scale. AI