Investigating Social Desirability Bias in Student Self-Report Surveys
Angie L. Miller
2012-09-01
Abstract:The frequent use of student self-report surveys in higher education calls into question the possibility of social desirability having an unwanted influence on responses. This research explores the potential presence of social desirability bias with the National Survey of Student Engagement (NSSE), a widely used assessment of student behaviors. Correlations between a short social desirability scale and NSSE benchmarks, subscales, and selected items suggest that the majority of scores have no significant relationship with a measure of social desirability. A series of regression models controlling for demographic variables produce similar results. Effect sizes and estimates of explained variance are also discussed.Survey research is becoming increasing common in education, in part due to heightened requirements for assessment and accountability at the institutional level (Kuh & Ewell, 2010). Surveys can provide a variety of information relatively easily, especially with the prevalence and availability of online data collection technology. However, this increased reliance on survey data calls into question the quality of the data. Can researchers and practitioners trust that the wealth of information gathered from the surveys is accurate and free from bias?If there is reason to believe that questions on a survey will prompt respondents to answer untruthfully in an attempt to provide a socially appropriate response, researchers may want to explore the potentiaT presence of social desirability bias (DeVellis, 2003). Many scales have been developed to measure the tendency to respond in a socially desirable manner (Crowne & Marlowe, 1960; Paulhus, 1984). Generally, these instruments are based on the premise that there are some "ideally good" behaviors that no one actually does, and there are some "ideally bad" behaviors that everyone actually does. For instance, if a person responds "yes" to the statement "I am always a good listener, no matter who I am talking to," this would be considered a socially desirable response. Even if a person is a much better listener than most, there have likely been occasions when he or she has been irritable, or distracted, or tired and was therefore not a good listener. Conversely, if a person responds "no" to the statement "I sometimes feel resentful when I don't get my own way" this would also be considered a socially desirable response. Based on this premise, measures of socially desirability are able to provide a score for one's tendency to respond in a socially desirable way, and this score can then be examined in conjunction with responses to other available instruments. If an instrument is free from social desirability bias, scores on the instrument should not be related to scores on a measure of social desirability.Investigations of social desirability bias were traditionally only conducted with surveys that contain items of a sensitive nature, such as sexual or drug-related behaviors (Carpenter, 2009). However, some research in higher education suggests that social desirability bias may have an impact on student responses for less sensitive topics as well. Recent studies have found significant relationships between social desirability and perceptions of institutional values, goal orientation, value commitment, major satisfaction, and self-reported gains with samples of undergraduate students (Bowman & Hill, 2011; Ferrari, Bristow, & Cowman, 2005; Ferrari & Cowman, 2004; Ferrari, McCarthy & Milner, 2009; Nauta, 2007). Given this broad array of areas that can be influenced by social desirability, it is important to further explore questions about the validity of using surveys with college students. These findings suggest that in higher education research, some self-report measures may be subject to social desirability bias.However, there is also research in higher education that fails to find significant relationships with social desirability. …
Education,Psychology