Exploring the potential of AI-Chatbots in organic chemistry: An assessment of ChatGPT and bard
Kassem Hallal,Rasha Hamdan,Sami Tlais
DOI: https://doi.org/10.1016/j.caeai.2023.100170
2023-10-12
Computers and Education: Artificial Intelligence
Abstract:The invention of AI-Chatbot is undeniably one of the most remarkable achievements by humanity, harnessing an unparalleled level of power and potential. In the near future, AI-chatbots are expected to become valuable tools in education, aiding students in their learning journeys. This study aims to explore the performance and accuracy of two chatbots, ChatGPT and Bard in understanding text-based structural notations such as, condensed structures, InChi and SMILES and answering organic chemistry related questions. Their ability to perform tasks such as converting IUPAC names, InChi, and SMILES notations into condensed forms and vice versa, identifying functional groups, generating molecular formulas, and predicting resonance patterns was studied. The ChatGPTs' accuracy percentages for various tasks are as follows: determining degree of unsaturation from molecular formulas (90-80%), InChi (79-64%), and SMILES (86-64%); identifying functional groups from condensed structures (94%) and InChi (65-50%); converting condensed structures to molecular formulas (86-73%), IUPAC (38%), InChi (22-17%), and SMILES (56-44%); converting InChi to IUPAC (65-50%) and condensed structures (28-11%); and converting SMILES to condensed structures (42-37%) and IUPAC (25-20%). In contrast, Bard consistently performed lower in most tasks. Both chatbots had significant limitations, especially with InChi and SMILES notations which have been used successfully in machine learning. GPT-4, the newer version of ChatGPT, was also tested against these tasks, and slight improvements were observed in most areas, particularly in reading SMILES notations. While these advanced AI-chatbots hold promising potential as enduring educational tools in organic chemistry, inspiring reevaluation of teaching strategies, their implementation should be carefully monitored, particularly considering the rapid pace of advancements.