A new research paper explores the capabilities and limitations of small open-weight language models in crowd decision-making (CDM). The study found that while these models can be run on private infrastructure, their judgments exhibit variability across repeated queries. Aggregating outputs can stabilize results but does not eliminate sensitivity to participant changes. Although larger open-weight models showed promise in matching a hosted reference's performance, no system could definitively name a single winner, instead reporting candidate sets with margins. AI
IMPACT Small open-weight models may enable localized crowd decision-making systems, though further development is needed for definitive outcomes.
RANK_REASON Research paper detailing capabilities and limits of LLMs for a specific task. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →