Using Placeholder Text in Narrative Open-Ended Questions in Web Surveys

Tanja Kunz,Franziska Quoß,Tobias Gummer

DOI: https://doi.org/10.1093/jssam/smaa039

2020-12-23

Journal of Survey Statistics and Methodology

Abstract:Abstract Narrative open-ended questions are suitable for gathering detailed information without limiting respondents to a predefined set of response categories. However, despite efforts to improve the quality of open-ended responses using different verbal and visual design features, respondents are often unwilling to expend effort on substantive and comprehensive responses. Based on a Web survey experiment conducted with opt-in panelists in Germany, we test whether placeholder text (i.e., lorem ipsum) in the answer box of a narrative open-ended question can be used as a visual stimulus to promote high-quality responses without discouraging respondents from answering the question. We find that, although placeholder texts that suggest long and extensive responses elicit more extensive responses, they also result in longer response times and less substantive responses. As the disadvantages of such lengthy placeholder texts thus appear to outweigh their advantages, we advise against using them. We further find that shorter placeholder texts do not provide any additional benefits. These findings also suggest that any kind of visual design feature should always be tested thoroughly before use.

statistics & probability,social sciences, mathematical methods