Researchers have developed a new conditional information retrieval framework to improve the harmonization of longitudinal survey data. This framework first extracts underlying sociological concepts from survey questions using a fine-tuned generative model and then uses these concepts as explicit conditioning signals for more accurate alignment. Tested on the CLOSER case study, the system consistently outperformed baseline methods by 2-3 points in both Mean Average Precision and Recall. The study also explored various fine-tuning strategies, finding that Supervised Fine-Tuning (SFT) provided the most stable improvements, and demonstrated that a model fine-tuned on a general social science thesaurus could effectively transfer to task-specific survey data, offering a practical solution for data-scarce scenarios. AI
IMPACT This framework could enhance the accuracy and efficiency of analyzing large-scale social science datasets by improving information retrieval for survey data.
RANK_REASON The item describes a new framework presented in a research paper on arXiv. [lever_c_demoted from research: ic=1 ai=1.0]
Read on arXiv cs.IR (Information Retrieval) →
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
