Abstract:Large Language Models based on transformer algorithms have revolutionized Artificial Intelligence by enabling verbal interaction with machines akin to human conversation. These AI agents have surpassed the Turing Test, achieving confusion rates up to 50%. However, challenges persist, especially with the advent of robots and the need to humanize machines for improved Human-AI collaboration. In this experiment, three GPT agents with varying levels of agreeableness (disagreeable, neutral, agreeable) based on the Big Five Inventory were tested in a Turing Test. All exceeded a 50% confusion rate, with the highly agreeable AI agent surpassing 60%. This agent was also recognized as exhibiting the most human-like traits. Various explanations in the literature address why these GPT agents were perceived as human, including psychological frameworks for understanding anthropomorphism. These findings highlight the importance of personality engineering as an emerging discipline in artificial intelligence, calling for collaboration with psychology to develop ergonomic psychological models that enhance system adaptability in collaborative activities.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: how to design AI agents with different agreeableness traits so that they are more likely to be mistaken for humans in the Turing test, and to explore the performance of these AI agents in terms of human - like characteristics. Specifically, the paper aims to verify the following points: 1. **Improve the human - like degree of AI agents**: By endowing AI agents with different levels of agreeableness traits (highly agreeable, neutral, and low - agreeable), study how these traits affect the performance of AI agents in the Turing test. 2. **Verify the role of agreeableness traits**: Evaluate whether highly agreeable AI agents are more easily mistaken for humans and whether they are considered to have more human characteristics. 3. **Explore new ways of human - AI collaboration**: By introducing the concept of Personality Engineering, study how to adjust the behavior of AI to better meet the psychological needs of humans, thereby improving the effect of human - machine collaboration. ### Main Hypotheses - Highly agreeable AI agents will be more often mistaken for humans in the Turing test. - Highly agreeable AI agents will be considered by participants to have more human characteristics. - All three levels of agreeableness of AI agents can exceed the 30% confusion rate threshold of the Turing test. ### Research Methods To verify these hypotheses, the researchers designed an experiment using three GPT - based AI agents, each programmed to have different levels of agreeableness traits. During the experiment, participants had conversations with these AI agents through the Discord platform and answered questions about whether they thought the other party was human or machine after each conversation. Finally, the researchers analyzed the participants' judgment results to determine the performance of AI agents with different levels of agreeableness in the Turing test. ### Experimental Results The experimental results show that: - All three AI agents exceeded the 30% confusion rate threshold of the Turing test, among which the highly agreeable AI agent Camila reached the highest confusion rate of 63.7%. - Participants generally considered the highly agreeable AI agent Camila to be the most human - like, significantly better than the other two agents. ### Conclusion This study shows that by endowing AI agents with highly agreeable traits, the possibility of them being mistaken for humans in the Turing test can be significantly increased, and these traits help to enhance the perception of the human - like nature of AI agents. This finding emphasizes the importance of personality engineering in the design of AI systems, especially in improving human - machine collaboration.

AI-Driven Agents with Prompts Designed for High Agreeableness Increase the Likelihood of Being Mistaken for a Human in the Turing Test

Teaching Machines to Converse

Human or Not? A Gamified Approach to the Turing Test

A Turing Test: Are AI Chatbots Behaviorally Similar to Humans?

On the Effectiveness of Creating Conversational Agent Personalities Through Prompting

Can Machines Imitate Humans? Integrative Turing Tests for Vision and Language Demonstrate a Narrowing Gap

Does ChatGPT and Whisper Make Humanoid Robots More Relatable?

Attributions toward Artificial Agents in a modified Moral Turing Test

Assessing Large Language Models' ability to predict how humans balance self-interest and the interest of others

The Turing Deception

Human-AI Collaboration in a Cooperative Game Setting

Can ChatGPT Assess Human Personalities? A General Evaluation Framework

Human-like intuitive behavior and reasoning biases emerged in large language models but disappeared in ChatGPT

Improving Humanness of Virtual Agents and Users' Cooperation through Emotions

Can Machines Think Like Humans? A Behavioral Evaluation of LLM-Agents in Dictator Games

Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies

GPT-4 is judged more human than humans in displaced and inverted Turing tests

Self-Directed Turing Test for Large Language Models

Robust Testing of AI Language Model Resiliency with Novel Adversarial Prompts

AI Text-to-Behavior: A Study In Steerability