Artificial Intelligence’s Capabilities, Limitations, and Impact on Accounting Education: Investigating ChatGPT’s Performance on Educational Accounting Cases

Ryan Dunn,Travis Holt,Kerry Inger,J. Gregory Jenkins,Jefferson Jones,James Long,Tina M. Loraas,Mollie Mathis,Jonathan D. Stanley,David A. Wood,Xu (Joyce) Cheng
DOI: https://doi.org/10.2139/ssrn.4431202
2023-01-01
SSRN Electronic Journal
Abstract:This study evaluates the capabilities of ChatGPT models 3.5 and 4 to provide solutions to eight educational accounting cases. We find that ChatGPT’s ability to provide accurate solutions varies depending on the type of case requirement and topical area of accounting, with better performance on tasks requiring elements that require explanation, application of rules, and ethical evaluation. However, ChatGPT performs poorly on tasks that require financial statement creation, journal entries, or software use. Our study also finds that detection tools provided by ChatGPT's developer are ineffective in identifying text created by Artificial Intelligence Text Generators (AITG). We suggest ways in which instructors can develop and structure assignments to reduce the effectiveness of AITGs in subverting the learning process, and discuss ways in which AITGs can be incorporated into assignment requirements to help students attain, desired educational outcomes.
What problem does this paper attempt to address?