Abstract:Large Language Models (LLMs) such as ChatGPT and GitHub Copilot have revolutionized automated code generation in software engineering. However, as these models are increasingly utilized for software development, concerns have arisen regarding the security and quality of the generated code. These concerns stem from LLMs being primarily trained on publicly available code repositories and internet-based textual data, which may contain insecure code. This presents a significant risk of perpetuating vulnerabilities in the generated code, creating potential attack vectors for exploitation by malicious actors. Our research aims to tackle these issues by introducing a framework for secure behavioral learning of LLMs through In-Content Learning (ICL) patterns during the code generation process, followed by rigorous security evaluations. To achieve this, we have selected four diverse LLMs for experimentation. We have evaluated these coding LLMs across three programming languages and identified security vulnerabilities and code smells. The code is generated through ICL with curated problem sets and undergoes rigorous security testing to evaluate the overall quality and trustworthiness of the generated code. Our research indicates that ICL-driven one-shot and few-shot learning patterns can enhance code security, reducing vulnerabilities in various programming scenarios. Developers and researchers should know that LLMs have a limited understanding of security principles. This may lead to security breaches when the generated code is deployed in production systems. Our research highlights LLMs are a potential source of new vulnerabilities to the software supply chain. It is important to consider this when using LLMs for code generation. This research article offers insights into improving LLM security and encourages proactive use of LLMs for code generation to ensure software system safety.

Saltzer & Schroeder for 2030: Security engineering principles in a world of AI

Software Security Analysis in 2030 and Beyond: A Research Roadmap

Generative AI for Cyber Security: Analyzing the Potential of ChatGPT, DALL-E, and Other Models for Enhancing the Security Space

From Today's Code to Tomorrow's Symphony: The AI Transformation of Developer's Routine by 2030

Embedding Secure Coding Instruction into the IDE: A Field Study in an Advanced CS Course.

Just another copy and paste? Comparing the security vulnerabilities of ChatGPT generated code and StackOverflow answers

Cybersecurity in the Age of Generative AI: Usable Security & Statistical Analysis of ThreatGPT

Using AI Assistants in Software Development: A Qualitative Study on Security Practices and Concerns

Better Call Saltzer \& Schroeder: A Retrospective Security Analysis of SolarWinds \& Log4j

Cyber Security Issues and Challenges Related to Generative AI and ChatGPT

Enhancing Security of AI-Based Code Synthesis with GitHub Copilot via Cheap and Efficient Prompt-Engineering

From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacy

SecCodePLT: A Unified Platform for Evaluating the Security of Code GenAI

With ChatGPT, do we have to rewrite our learning objectives -- CASE study in Cybersecurity

Is Your AI-Generated Code Really Safe? Evaluating Large Language Models on Secure Code Generation with CodeSecEval

Can We Trust Large Language Models Generated Code? A Framework for In-Context Learning, Security Patterns, and Code Evaluations Across Diverse LLMs

A Marauder's Map of Security and Privacy in Machine Learning

Do Users Write More Insecure Code with AI Assistants?

Security Challenges in Autonomous Systems Design

AI Code Generators for Security: Friend or Foe?