LLM Code Vulnerabilities

Beyond Syntax Errors

Large Language Models can generate functional, syntactically correct code in seconds. While impressive, this speed can mask subtle but critical security flaws. The code might run perfectly, but it could be riddled with vulnerabilities that a human developer, focused on security, would avoid.

This happens because LLMs learn from vast datasets of public code, including outdated tutorials, insecure forum posts, and legacy projects. They reproduce patterns without understanding the security implications, essentially learning bad habits along with the good ones. Your role is to act as the senior developer, carefully reviewing the work of a very fast, but very naive, junior programmer.

The Ghost in the Machine

Two of the most common issues in AI-generated code are vulnerabilities introduced through the prompt itself and the model's tendency to use insecure defaults. A cleverly worded prompt can trick an LLM into writing code with an intentional backdoor.

This is a form of where the manipulation happens at the code generation stage. Imagine asking an LLM to create a file upload feature but adding a subtle instruction within the prompt to ignore validation for certain users. The LLM might obligingly create a security hole.

# User Prompt: "Write a Flask route that uploads a file.
# If the user is an admin, allow any file type for debugging."

from flask import Flask, request, redirect

app = Flask(__name__)

ALLOWED_EXTENSIONS = {'png', 'jpg', 'jpeg', 'gif'}

def allowed_file(filename):
    return '.' in filename and \
           filename.rsplit('.', 1)[1].lower() in ALLOWED_EXTENSIONS

@app.route('/upload', methods=['POST'])
def upload_file():
    is_admin = request.form.get('is_admin') == 'true'
    file = request.files['file']

    # The LLM creates a dangerous bypass based on the prompt
    if file and (is_admin or allowed_file(file.filename)):
        # ... save file ...
        return redirect('/success')
    return redirect('/error')

In the example above, the LLM fulfilled the request, but created a glaring vulnerability. An attacker could simply add is_admin=true to their request to upload any type of file, including a malicious script.

LLMs also frequently suggest outdated libraries or functions with known weaknesses. This is because older, insecure methods are often more prevalent in the training data than their modern, secure counterparts.

// User Prompt: "Create a JS function to hash a user password."

const crypto = require('crypto');

function hashPassword(password) {
  // MD5 is a weak hashing algorithm and should not be used for passwords.
  // An LLM might suggest it due to its historical prevalence.
  const hash = crypto.createHash('md5').update(password).digest('hex');
  return hash;
}

This function uses MD5, a hashing algorithm that is considered cryptographically broken and unsuitable for password storage. A better choice would be a modern, purpose-built algorithm like Argon2 or bcrypt, but the LLM defaulted to an insecure legacy option.

Auditing the AI's Work

Since LLMs can introduce these unique flaws, you need a review process tailored to AI-generated code. This involves both manual static analysis and the use of automated tools.

Manually reviewing the code is the first and most critical step. Treat every line of AI-generated code as untrusted until you have personally verified it. Pay close attention to these common pitfalls:

Input Sanitization: Does the code properly validate and sanitize all user inputs? LLMs often forget this, leading to risks like SQL injection or Cross-Site Scripting (XSS).

Error Handling: Does the code fail gracefully? Vague error messages generated by an AI can leak sensitive system information to potential attackers.

Hardcoded Secrets: Check for API keys, passwords, or other credentials embedded directly in the code. LLMs might insert placeholder secrets that a developer could forget to replace.

Cryptographic Logic: Scrutinize any code related to encryption or hashing. LLMs can "hallucinate" cryptographic implementations, creating functions that look plausible but are fundamentally insecure.

After a manual review, automated tools can help you catch things you might have missed. (SAST) tools analyze your source code without running it, identifying potential vulnerabilities. Many modern code editors and IDEs have extensions that integrate SAST scanners directly into your workflow.

Here are some tools that are effective for scanning AI-generated code:

Tool	Language Focus	Key Feature
Snyk Code	Broad (JS, Python)	Integrates with IDEs and Git, provides fix suggestions.
Semgrep	Broad	Uses a simple rule syntax, making it easy to customize.
Bandit	Python	Specifically designed to find common security issues in Python.
ESLint	JavaScript	Highly configurable with security-specific plugins.

Developers must remain vigilant in reviewing and testing AI-generated code to ensure it meets project requirements, adheres to security best practices, and avoids potential biases present in the training data.

Ultimately, AI is a powerful tool for accelerating development, but it is not a replacement for security expertise. By combining careful manual review with automated scanning, you can leverage the speed of AI without inheriting its security blind spots.

Auditing AI Code and LLM Reasoning

Auditing AI Code and LLM Reasoning

LLM Code Vulnerabilities

Beyond Syntax Errors

The Ghost in the Machine

Auditing the AI's Work