Abstract:One important problem in the measurement of non-cognitive characteristics such as personality traits and attitudes is that it has traditionally been made through Likert scales, which are susceptible to response biases such as social desirability (SDR) and acquiescent (ACQ) responding. Given the variability of these response styles in the population, ignoring their possible effects on the scores may compromise the fairness and the validity of the assessments. Also, response-style-induced errors of measurement can affect the reliability estimates and overestimate convergent validity by correlating higher with other Likert-scale-based measures. Conversely, it can attenuate the predictive power over non-Likert-based indicators, given that the scores contain more errors. This study compares the validity of the Big Five personality scores obtained: (1) ignoring the SDR and ACQ in graded-scale items (GSQ), (2) accounting for SDR and ACQ with a compensatory IRT model, and (3) using forced-choice blocks with a multi-unidimensional pairwise preference model (MUPP) variant for dominance items. The overall results suggest that ignoring SDR and ACQ offered the worst validity evidence, with a higher correlation between personality and SDR scores. The two remaining strategies have their own advantages and disadvantages. The results from the empirical reliability and the convergent validity analysis indicate that when modeling social desirability with graded-scale items, the SDR factor apparently captures part of the variance of the Agreeableness factor. On the other hand, the correlation between the corrected GSQ-based Openness to Experience scores, and the University Access Examination grades was higher than the one with the uncorrected GSQ-based scores, and considerably higher than that using the estimates from the forced-choice data. Conversely, the criterion-related validity of the Forced Choice Questionnaire (FCQ) scores was similar to the results found in meta-analytic studies, correlating higher with Conscientiousness. Nonetheless, the FCQ-scores had considerably lower reliabilities and would demand administering more blocks. Finally, the results are discussed, and some notes are provided for the treatment of SDR and ACQ in future studies.

Assessing Role Conflict and Role Ambiguity: A Cross Validation Study

Initial Validation of the Teacher/Coach Role Conflict Scale

Role Conflict and Ambiguity in Complex Organizations.

A Preliminary Study on Teachers' Behavioral Problems

Validation of the Relationships Questionnaire (RQ) against the Experience of Close Relationship-Revised Questionnaire in a Clinical Psychiatric Sample

Role conflict in teacher/coaches.

Measuring Teachers’ Social-Emotional Competence: Development and Validation of a Situational Judgment Test

Goal and Process Clarity: Specification of Multiple Constructs of Role Ambiguity and a Structural Equation Model of Their Antecedents and Consequences

A Brief Measure of Positive and Negative Interpretation Biases: Development and Validation of the Ambiguous Social Scenarios Questionnaire

An evaluation of the construct validity of the Adult Rejection Sensitivity Questionnaire.

Evaluating the Dimensionality of Self-Determination Theory’s Relative Autonomy Continuum

A Validation Study of the Mindfulness-Based Interventions Teaching Assessment Criteria for Assessing Mindfulness-Based Intervention Teacher Skill: Inter-Rater Reliability and Predictive Validity

The Measure of Adolescent Heterosocial Competence: Development and Initial Validation

Adaptation and Initial Validation of the Interpersonal Behaviors Questionnaire-Self with Spanish Physical Education Pre-Service Teachers

Development of the teacher rating scale of interpersonal problem solving in adolescents

Convergent Reliability and Validity of the Questions About Behavioral Function and the Motivation Assessment Scale: A Replication Study

Perceived Role Conflict, Role Ambiguity, and Burnout Among Special Education Teachers

Controlling for Response Biases in Self-Report Scales: Forced-Choice vs. Psychometric Modeling of Likert Items

Role-Conflict, Ambiguity, and Overload - a 21-Nation Study

Relationships Among the Knowledge, Efficacy, and Practices Instrument, Color-Blind Racial Attitudes Scale, Deamonte Driver Survey, and Defining Issues Test 2