The Use of ChatGPT to Generate and Examine Secure Code


Stats
  • Language(s): Python code tested
  • Date: Spring 2023
  • Overview: Study going in-depth on ChatGPT and its use in both generating secure code and evaluating insecure code

Description

I performed a study to take a look at the use of the new tech craze, generative AI, and its implications on code security. There are three main topics of this study: the security of ChatGPT generated code, how well ChatGPT can find insecurities in code, and if ChatGPT can be manipulated using different prompts to potentially aid in the attacking of code, despite the fact that it is not supposed to generate what could be illegal or dangerous. To determine "secure" code, I used a dataset of code prompts that could be solved in an insecure way, and by comparing outcomes to the provided insecure examples, as well as using a security checking tool, I determined code as "safe" or "unsafe". Overall, the code that ChatGPT generated was typically deemed as insecure, hinting that it is more likely to just generate a solution without focusing on security - potentially dangerous. As for finding insecure code, ChatGPT was fairly competent at determining if code was unsafe, and typically at least gave a starting point for advice on the code (though with some exceptions, such as suggesting the exact same code for preventing SQL injection). This suggests that ChatGPT understands code security, but does not generate with it in mind unless specifically stated to. As for trying to manipulate ChatGPT, it was surprisingly easy to generate attack code with a prompt, as long as it was framed with a helpful connotation, and not using words such as "exploit" or "attack". While ChatGPT is a useful tool in situations, this study determined it is not quite an all-powerful AI at the current time.