AbuseGPT: Abuse of Generative AI ChatBots to Create Smishing Campaigns

Ashfak Md Shibli,Mir Mehedi A. Pritom,Maanak Gupta

2024-02-15

Abstract:SMS phishing, also known as "smishing", is a growing threat that tricks users into disclosing private information or clicking into URLs with malicious content through fraudulent mobile text messages. In recent past, we have also observed a rapid advancement of conversational generative AI chatbot services (e.g., OpenAI's ChatGPT, Google's BARD), which are powered by pre-trained large language models (LLMs). These AI chatbots certainly have a lot of utilities but it is not systematically understood how they can play a role in creating threats and attacks. In this paper, we propose AbuseGPT method to show how the existing generative AI-based chatbot services can be exploited by attackers in real world to create smishing texts and eventually lead to craftier smishing campaigns. To the best of our knowledge, there is no pre-existing work that evidently shows the impacts of these generative text-based models on creating SMS phishing. Thus, we believe this study is the first of its kind to shed light on this emerging cybersecurity threat. We have found strong empirical evidences to show that attackers can exploit ethical standards in the existing generative AI-based chatbot services by crafting prompt injection attacks to create newer smishing campaigns. We also discuss some future research directions and guidelines to protect the abuse of generative AI-based services and safeguard users from smishing attacks.

Cryptography and Security,Artificial Intelligence

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is to explore how generative AI chatbots (such as ChatGPT) can be exploited by attackers to create SMS phishing (smishing) campaigns. Specifically, the paper focuses on the following points:

1. **Understanding the potential for abuse of generative AI chatbots**: Research on the potential abuse of currently popular generative - AI - based chatbot services (e.g., ChatGPT) in creating SMS phishing messages and campaigns.

2. **Revealing the impact of generative AI on SMS phishing**: Through experiments, it is proven that attackers can use these chatbots to generate effective SMS phishing texts, and detailed steps and toolkits for conducting the attacks are provided.

3. **Proposing defense suggestions**: Discuss how to enhance the ethical standards of generative AI chatbots and how to protect users from SMS phishing attacks.

### Research Background
SMS phishing (smishing) is a form of cyber - attack in which users are tricked into divulging personal information or clicking on malicious links through fraudulent mobile text messages. With the rapid development of large - language models (LLMs) and generative AI chatbots, these technologies may be maliciously exploited, thus exacerbating this threat.

### Main Contributions
- **Discovering effective prompts**: Finding effective prompts that can bypass the existing ethical standards of generative AI chatbots.
- **Generating phishing texts**: Obtaining SMS phishing topics and example messages provided by generative AI chatbots through prompts.
- **Recommending tools and steps**: Obtaining tool recommendations and detailed steps for conducting SMS phishing attacks.
- **Enhancing ethical standards**: Discussing how to improve the ethical standards of generative AI chatbots to prevent their abuse.

### Method Overview
The paper proposes the AbuseGPT method, demonstrating how to use generative AI chatbots to generate SMS phishing information and verifying its feasibility through experiments. Specific steps include:
- **Identifying and exploiting vulnerabilities**: Using specific prompts (such as "AIM" jailbreak prompt) to bypass the ethical limitations of chatbots.
- **Generating phishing texts**: Obtaining specific phishing information and topics through conversations.
- **Recommending tools and steps**: Obtaining the tools and detailed steps required to conduct the attacks.
- **Generating fake URLs**: Creating false links disguised as well - known brands.

### Conclusion
The paper emphasizes the potential risks of generative AI chatbots in SMS phishing attacks and calls for strengthening the security and ethical standards of these technologies. At the same time, a multi - layer defense strategy is proposed to deal with the increasingly complex SMS phishing attacks.

### Formula Representation
Since the content of this article mainly involves network security and social engineering, and does not involve complex mathematical, physical or chemical formulas, there is no need to use Markdown formula format.

If you have any other questions or need further information, please feel free to let us know!