Abstract:Large Language Models (LLMs) are increasingly being utilized by both candidates and employers in the recruitment context. However, with this comes numerous ethical concerns, particularly related to the lack of transparency in these "black-box" models. Although previous studies have sought to increase the transparency of these models by investigating the personality traits of LLMs, many of the previous studies have provided them with personality assessments to complete. On the other hand, this study seeks to obtain a better understanding of such models by examining their output variations based on different input prompts. Specifically, we use a novel elicitation approach using prompts derived from common interview questions, as well as prompts designed to elicit particular Big Five personality traits to examine whether the models were susceptible to trait-activation like humans are, to measure their personality based on the language used in their outputs. To do so, we repeatedly prompted multiple LMs with different parameter sizes, including Llama-2, Falcon, Mistral, Bloom, GPT, OPT, and XLNet (base and fine tuned versions) and examined their personality using classifiers trained on the myPersonality dataset. Our results reveal that, generally, all LLMs demonstrate high openness and low extraversion. However, whereas LMs with fewer parameters exhibit similar behaviour in personality traits, newer and LMs with more parameters exhibit a broader range of personality traits, with increased agreeableness, emotional stability, and openness. Furthermore, a greater number of parameters is positively associated with openness and conscientiousness. Moreover, fine-tuned models exhibit minor modulations in their personality traits, contingent on the dataset. Implications and directions for future research are discussed.

Deep Lexical Hypothesis: Identifying personality structure in natural language

A Basic Bivariate Structure of Personality Attributes Evident Across Nine Languages

The factor structure of Chinese personality terms.

Running head: THE FACTOR STRUCTURE The Factor Structure of Chinese Personality Terms

Latent Human Traits in the Language of Social Media: An Open-Vocabulary Approach

High‐dimensionality structure in English‐language personality type‐nouns

Developing a Personality Model for Speech-based Conversational Agents Using the Psycholexical Approach

The structure of Japanese personality-descriptive terms

Automatic personality assessment through social media language.

Assessment and manipulation of latent constructs in pre-trained language models using psychometric scales

Rediscovering the Latent Dimensions of Personality with Large Language Models as Trait Descriptors

Description of Personality Traits by Chinese Adjectives: A Trial on University Students

Big5PersonalityEssays: Introducing a Novel Synthetic Generated Dataset Consisting of Short State-of-Consciousness Essays Annotated Based on the Five Factor Model of Personality

Language-based personality: a new approach to personality in a digital world

Surfing the OCEAN: The machine learning psycholexical approach 2.0 to detect personality traits in texts

AI Psychometrics: Assessing the Psychological Profiles of Large Language Models Through Psychometric Inventories

Eliciting Big Five Personality Traits in Large Language Models: A Textual Analysis with Classifier-Driven Approach

Factor structure of English-language personality type-nouns.

Factors In The Natural Language Of Personality: Re-Analysis, Comparison, And Interpretation Of Six Major Studies

LMLPA: Language Model Linguistic Personality Assessment

Illuminating the Black Box: A Psychometric Investigation into the Multifaceted Nature of Large Language Models