LLMs struggle with tabular feature selection compared to classical methods

A new empirical study on arXiv investigates the effectiveness of Large Language Models (LLMs) in tabular feature selection. The research found that classical statistical methods generally outperform LLM-based selectors, including models like Qwen2.5-7B and Llama-3.3-70B, in terms of predictive performance. While LLMs can leverage semantic feature names, this benefit was more pronounced in regression tasks than in classification, and dataset-dependent. The study concludes that traditional selectors remain strong baselines for predictive tabular feature selection. AI

IMPACT Suggests classical methods remain superior for tabular feature selection, indicating LLMs may not be a universal solution for all data analysis tasks.

RANK_REASON Academic paper published on arXiv detailing empirical study of LLM performance on feature selection. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• arXiv
• Hugging Face
• Llama-3.3-70B
• LLMs
• Qwen2.5-7B
• Qwen 7B
• ROC-AUC

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →