Abstract:One important problem in the measurement of non-cognitive characteristics such as personality traits and attitudes is that it has traditionally been made through Likert scales, which are susceptible to response biases such as social desirability (SDR) and acquiescent (ACQ) responding. Given the variability of these response styles in the population, ignoring their possible effects on the scores may compromise the fairness and the validity of the assessments. Also, response-style-induced errors of measurement can affect the reliability estimates and overestimate convergent validity by correlating higher with other Likert-scale-based measures. Conversely, it can attenuate the predictive power over non-Likert-based indicators, given that the scores contain more errors. This study compares the validity of the Big Five personality scores obtained: (1) ignoring the SDR and ACQ in graded-scale items (GSQ), (2) accounting for SDR and ACQ with a compensatory IRT model, and (3) using forced-choice blocks with a multi-unidimensional pairwise preference model (MUPP) variant for dominance items. The overall results suggest that ignoring SDR and ACQ offered the worst validity evidence, with a higher correlation between personality and SDR scores. The two remaining strategies have their own advantages and disadvantages. The results from the empirical reliability and the convergent validity analysis indicate that when modeling social desirability with graded-scale items, the SDR factor apparently captures part of the variance of the Agreeableness factor. On the other hand, the correlation between the corrected GSQ-based Openness to Experience scores, and the University Access Examination grades was higher than the one with the uncorrected GSQ-based scores, and considerably higher than that using the estimates from the forced-choice data. Conversely, the criterion-related validity of the Forced Choice Questionnaire (FCQ) scores was similar to the results found in meta-analytic studies, correlating higher with Conscientiousness. Nonetheless, the FCQ-scores had considerably lower reliabilities and would demand administering more blocks. Finally, the results are discussed, and some notes are provided for the treatment of SDR and ACQ in future studies.

Testing the latent structure, factorial equivalence, and external correlates of the brief self-control scale in a community sample of Spanish adults

Brief multidimensional self-control scale: psychometric properties and cross-gender measurement invariance of the Portuguese version

Big Five Personality Trait Short Questionnaire: Preliminary Validation with Spanish Adults

Factor structure, gender invariance, measurement properties and short forms of the Spanish adaptation of the Big Five Inventory-2 (BFI-2)

Reliability Generalization Meta-Analysis of the Brief Self-Control Scale (BSCS): Reliability Evidence across Age Groups and Languages

Controlling for Response Biases in Self-Report Scales: Forced-Choice vs. Psychometric Modeling of Likert Items

Brief Version of the Revised-Abbreviated Eysenck Personality Questionnaire in a Spanish Young Adult Population

Self- vs. External-Regulation Behavior ScaleTM in different psychological contexts: A validation study

The Toronto Mindfulness Scale and the State Mindfulness Scale: psychometric properties of the Spanish versions

The Behavioral, Emotional, and Social Skills Inventory: A Spanish Adaptation and Further Validation in Adult Population

The Multidimensional Self-Control Scale (MSCS): Development and validation.

Psychometric properties and Longitudinal Measurement Invariance of the Spanish version of the Alcohol Expectancies Questionnaire Short Form among young adult binge drinkers

Motivational Influences on Health, Well-Being, and Lifestyle: Validation of the Spanish Version of the Treatment Self-Regulation Questionnaire in Four Health Domains

Spanish validation of the pure procrastination scale: dimensional structure, internal consistency, temporal stability, gender invariance, and relationships with personality and satisfaction with life

Spanish Adaptation and Psychometric Properties of the Protective Behavioral Strategies for Marijuana Scale: The Protective Behavioral Strategies for Cannabis Scale (S-PBSC)

Measurement Invariance of the Prosocial Behavior Scale in Three Hispanic Countries (Argentina, Spain, and Peru)

Validation of the Spanish version of the Bergen Social Media Addiction Scale (BSMAS) among Spanish adolescents

Spanish Version of the Bedtime Procrastination Scale: Cross-Cultural Adaptation and Psychometric Evaluation in a Sample of Nursing Students

Factorial Structure and Invariance Analysis of the BIS–BAS Scale, IPIP-R Version

Preliminary validity and reliability evidence of the Brief Antisocial Behavior Scale (B-ABS) in young adults from four countries

The assessment of emotional and Behavioural problems: Internal structure of The Strengths and Difficulties Questionnaire