Abstract:One important problem in the measurement of non-cognitive characteristics such as personality traits and attitudes is that it has traditionally been made through Likert scales, which are susceptible to response biases such as social desirability (SDR) and acquiescent (ACQ) responding. Given the variability of these response styles in the population, ignoring their possible effects on the scores may compromise the fairness and the validity of the assessments. Also, response-style-induced errors of measurement can affect the reliability estimates and overestimate convergent validity by correlating higher with other Likert-scale-based measures. Conversely, it can attenuate the predictive power over non-Likert-based indicators, given that the scores contain more errors. This study compares the validity of the Big Five personality scores obtained: (1) ignoring the SDR and ACQ in graded-scale items (GSQ), (2) accounting for SDR and ACQ with a compensatory IRT model, and (3) using forced-choice blocks with a multi-unidimensional pairwise preference model (MUPP) variant for dominance items. The overall results suggest that ignoring SDR and ACQ offered the worst validity evidence, with a higher correlation between personality and SDR scores. The two remaining strategies have their own advantages and disadvantages. The results from the empirical reliability and the convergent validity analysis indicate that when modeling social desirability with graded-scale items, the SDR factor apparently captures part of the variance of the Agreeableness factor. On the other hand, the correlation between the corrected GSQ-based Openness to Experience scores, and the University Access Examination grades was higher than the one with the uncorrected GSQ-based scores, and considerably higher than that using the estimates from the forced-choice data. Conversely, the criterion-related validity of the Forced Choice Questionnaire (FCQ) scores was similar to the results found in meta-analytic studies, correlating higher with Conscientiousness. Nonetheless, the FCQ-scores had considerably lower reliabilities and would demand administering more blocks. Finally, the results are discussed, and some notes are provided for the treatment of SDR and ACQ in future studies.

A less evaluative measure of Big Five personality: Comparison of structure and criterion validity

Less Evaluative Measures of Personality in Job Applicant Contexts: The Effect on Socially Desirable Responding and Criterion Validity

Testing the Measurement Equivalence of Personality Adjective Items Across Cultures

Validation of an abbreviated Big Five personality inventory at large population scale: Psychometric structure and associations with common psychiatric and neurological disorders

The Mini-IPIP Scales: Tiny-yet-effective measures of the Big Five Factors of Personality.

Though Forced, Still Valid: Examining the Psychometric Performance of Forced-Choice Measurement of Personality in Children and Adolescents

Evaluating a Brief Big Five Personality Test in a Diverse Chinese Sample: The Role of Midpoint Designs and Reversely-Worded Items

Validating a brief measure of four facets of social evaluation

Item subtlety and face validity in personality assessment.

Controlling for Response Biases in Self-Report Scales: Forced-Choice vs. Psychometric Modeling of Likert Items

Normal personality assessment in clinical practice: The NEO Personality Inventory.

On the Usefulness of Measures of Normal Personality for Clinical Assessment: Evidence of the Incremental Validity of the Revised NEO Personality Inventory.

Validation of the five-factor model of personality across instruments and observers.

The Structure Among Measures of Personality, Social Attitudes, Values, and Social Norms

Big Five Personality Trait Short Questionnaire: Preliminary Validation with Spanish Adults

A little garbage in, lots of garbage out: Assessing the impact of careless responding in personality survey data

Validation of the BFI-10-R: A New BFI Scale with Strong Structural and Construct Validity

The dominance of introspective measures and what this implies: The example of environmental attitude

How do we know we are measuring environmental attitude? Specific objectivity as the formal validation criterion for measures of latent attributes

Performance of the Personality Inventory for Youth validity scales

Testing whether the DSM-5 personality disorder trait model can be measured with a reduced set of items: An item response theory investigation of the Personality Inventory for DSM-5.