Researchers have introduced TabQueryBench, a new benchmark designed to evaluate the fidelity of synthetic tabular data specifically for analytical queries. Unlike existing methods that focus on statistical similarity, TabQueryBench uses SQL queries to assess how well synthetic data preserves the structure needed for data analysis. Experiments across numerous datasets and generative models revealed that current models, while good at statistical similarity, still fall short in query-centric fidelity, with RealTabFormer showing the best performance but still only achieving 0.75 fidelity. The benchmark also highlighted challenges for generative models with high-cardinality data and a trade-off between fidelity and generation cost, where BayesNet offers a strong balance. AI
IMPACT Introduces a new evaluation standard for synthetic data generation, potentially improving the utility of synthetic datasets for analytical tasks.
RANK_REASON The item is a research paper introducing a new benchmark for evaluating synthetic data. [lever_c_demoted from research: ic=1 ai=1.0]
- alphaXiv
- arXiv
- BayesNet
- CatalyzeX
- DagsHub
- Gotit.pub
- Hugging Face
- RealTabFormer
- ScienceCast
- SQL
- TabQueryBench
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
