Paper Alert: SODA @ ACL 2026: Simulating Survey Responses with LLMs

ACL is the leading venue for research in computational linguistics and natural language processing. This year's SODA contribution takes up a question that sits between the two fields and survey methodology: how researchers should go about simulating human survey responses with large language models.

How You Ask the Model Matters
Georg Ahnert, Anna-Carolina Haensch, Prof. Dr. Barbara Plank, and Prof. Dr. Markus Strohmaier presented "Survey Response Generation: Generating Closed-Ended Survey Responses In-Silico with Large Language Models". Language models are trained to produce open-ended text, but survey simulation requires them to pick one of a fixed set of answer options. No standard practice has emerged for how to bridge that gap.

The authors compared eight generation methods across four political attitude surveys and ten open-weight models, resulting in 32 million simulated responses. The choice of method produced substantial differences in how closely the simulated answers matched real ones, both for individual respondents and at the subpopulation level. Restricted generation methods performed best overall, while reasoning output did not reliably improve alignment. The paper closes with recommendations for applied work.