Abstract:Backgrounds: The field of Artificial Intelligence (AI) has seen a major shift in recent years due to the development of new Machine Learning (ML) models such as Generative Pre-trained Transformer (GPT). GPT has achieved previously unheard-of levels of accuracy in most computerized language processing tasks and their chat-based variations. Aim: The aim of this study was to investigate the problem-solving abilities of ChatGPT using two sets of verbal insight problems, with a known performance level established by a sample of human participants. Materials and methods: A total of 30 problems labeled as "practice problems" and "transfer problems" were administered to ChatGPT. ChatGPT's answers received a score of "0" for each incorrectly answered problem and a score of "1" for each correct response. The highest possible score for both the practice and transfer problems was 15 out of 15. The solution rate for each problem (based on a sample of 20 subjects) was used to assess and compare the performance of ChatGPT with that of human subjects. Results: The study highlighted that ChatGPT can be trained in out-of-the-box thinking and demonstrated potential in solving verbal insight problems. The global performance of ChatGPT equalled the most probable outcome for the human sample in both practice problems and transfer problems as well as upon their combination. Additionally, ChatGPT answer combinations were among the 5% of most probable outcomes for the human sample both when considering practice problems and pooled problem sets. These findings demonstrate that ChatGPT performance on both set of problems was in line with the mean rate of success of human subjects, indicating that it performed reasonably well. Conclusions: The use of transformer architecture and self-attention in ChatGPT may have helped to prioritize inputs while predicting, contributing to its potential in verbal insight problem-solving. ChatGPT has shown potential in solving insight problems, thus highlighting the importance of incorporating AI into psychological research. However, it is acknowledged that there are still open challenges. Indeed, further research is required to fully understand AI's capabilities and limitations in verbal problem-solving.

AI-based chatbot interactions and critical thinking skills: an exploratory study

Empowering Language Learners’ Critical Thinking: Evaluating ChatGPT’s Role in English Course Implementation

Artificial human thinking: ChatGPT’s capacity to be a model for critical thinking when prompted with problem-based writing activities

Use of ChatGPT at University as a Tool for Complex Thinking: Students' Perceived Usefulness

Befriending ChatGPT and other superchatbots: An AI-integrated take-home assessment preserving integrity

Enhancing Critical Thinking in Education by means of a Socratic Chatbot

The influence of AI text generators on critical thinking skills in UK business schools

AI and Education: An Investigation into the Use of ChatGPT for Systems Thinking

The advantages and limitations of using ChatGPT to enhance technological research

Enhancing STEM Learning with ChatGPT and Bing Chat as Objects to Think With: A Case Study

What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education

A framework and exemplars for ethical and responsible use of AI Chatbot technology to support teaching and learning

ChatGPT: A Study of AI Language Processing and its Implications

Critical Role of Artificially Intelligent Conversational Chatbot

Human-like problem-solving abilities in large language models using ChatGPT

ChatGPT in higher education: factors influencing ChatGPT user satisfaction and continued use intention

ChatGPT Goes to College: Exploring Student Perspectives on Artificial Intelligence in the Classroom

Are users willing to embrace ChatGPT? Exploring the factors on the acceptance of chatbots from the perspective of AIDUA framework

Artificial Intelligence in Healthcare Education: Investigating ChatGPT's Acceptance

Enhancing Human-Computer Interaction through AI: A Study on ChatGPT in Educational Environments

Using ChatGPT in the Development of Clinical Reasoning Cases: A Qualitative Study