LLMTabBench: Evaluating LLMs on Binary Tabular Classification From Zero to Few Shots
Researchers have introduced LLMTabBench, a new benchmark designed to evaluate how well Large Language Models (LLMs) perform on binary tabular classification tasks with limited data. The benchmark reveals that LLMs can be competitive in zero-shot scenarios, sometimes outperforming models that use few-shot examples. However, adding more few-shot examples can sometimes hinder LLM performance due to conflicts with their existing knowledge, and performance degrades with increasing data complexity. AI
IMPACT Provides a framework for understanding LLM capabilities and limitations in tabular data tasks, guiding deployment in low-data scenarios.