Project Description
Online surveys are prominent in many different disciplines. Despite the best efforts of questionnaire designers, participants regularly respond in unintended ways. In web surveys there are no interviewers to help respondents navigate difficulties or misunderstandings. However, respondents leave clues in keystrokes, response times, and mouse movements reflecting their challenges during the measurement process (Couper, 1998). Such paradata, collected alongside the primary data collection, can help detect breakdowns in survey design and eventually help improve it (Kreuter, 2013).
The overarching goal of this project is the improvement of web survey data collection. More specifically, we focus on developing widely applicable paradata measures based on cursor data, including click data produced by mouse movements or equivalents (fingers or a stylus on tablets) but also keyboard input, and changes to the questionnaire field. With these, we seek to show the practical usefulness of paradata in the field, beyond small-scale pilot studies – such as our studies in the first funding period – that have already demonstrated its utility. At the end of the project period, survey practitioners will have an easy-to-use toolkit at their disposal that solves a number of problems the field is facing, including detection of problematic items and participants, bots and inattentive respondents.
Contact Person
Publications
- Fernández-Fontelo, A., F. Henninger, P. J. Kieslich, F. Kreuter, and S. Greven (2020). A new
model for multivariate functional data classication with application to the prediction of diffi-
culty in web surveys using mouse movement trajectories. In Proceedings of the 35th Inter-
national Workshop on Statistical Modelling: July 20-24, 2020 Bilbao, Basque Country, Spain,
pp. 73–78. - Fernández-Fontelo, A., P. J. Kieslich, F. Henninger, F. Kreuter, and S. Greven (2021). Pre-
dicting question difficulty in web surveys: A machine-learning approach based on mouse
movement features. Social Science Computer Review, online first, https://doi.org/10. 1177/08944393211032950. - Henninger, F. and P. J. Kieslich (2022). Mousetrap-web: Mouse-Tracking for the browser.
Behavior Research Methods, to appear. - Henninger, F., Y. Shevchenko, U. K. Mertens, P. J. Kieslich, and B. E. Hilbig (2022). lab.js: A
free, open, online study builder. Behavior Research Methods 54(2), 556–573. - Keusch, F. and F. Kreuter (2021). Digital trace data. In U. Engel, A. Quan-Haase, S. X. Liu,
and L. Lyberg (Eds.), Handbook of Computational Social Science, Volume 1, pp. 100–118.
Taylor & Francis. - Kieslich, P. J., F. Henninger, D. U. Wulff, J. M. B. Haslbeck, and M. Schulte-Mecklenbeck (2019).
Mouse-tracking: A practical guide to implementation and analysis. In A Handbook of Process
Tracing Methods (Second ed.)., pp. Ebook 20. Routledge - Fernández-Fontelo, A., F. Henninger, P. J. Kieslich, F. Kreuter, and S. Greven (2022). Classi-
fication ensembles for multivariate functional data with application to mouse movements in
web surveys. Technical Report arXiv:2205.13380, https://arxiv.org/abs/2205.13380. - Henninger, F., P. J. Kieslich, A. Fernández-Fontelo, S. Greven, and F. Kreuter (2022). Privacy
attitudes toward mouse-tracking paradata collection. Preprint, SocArXiv, 10.31235/osf.io/
6weqx. - Leipold, F., P. J. Kieslich, F. Henninger, A. Fernández-Fontelo, S. Greven, and
F. Kreuter (2022). Detecting respondent burden in online surveys: How different
sources of question difficulty influence cursor movements. Manuscript under review,
https://osf.io/v69uk?view_only=aa2038ed9c014a30a987d3defa699dc1. - Wulff, D. U., P. J. Kieslich, F. Henninger, J. M. B. Haslbeck, and M. Schulte-Mecklenbeck (2021).
Movement tracking of cognitive processes: A tutorial using mousetrap. Preprint, PsyArXiv,
10.31234/osf.io/v685r.