• Browse topics
Sign up



For Snyk Enterprise customers with regional contracts. More info

Sign up

Insecure plugins for LLMs

External plugins bring extra functionality and extra danger


Insecure plugin: the basics

An insecure plugin refers to a software component or extension that contains vulnerabilities or weaknesses that could potentially be exploited by attackers to compromise the security of a system or application. These plugins are often used to add functionality or features to a larger software platform or system, but if they're poorly designed or implemented, they can introduce security risks.

In the context of web browsers, plugins or browser extensions can be examples of insecure plugins if they have known security flaws or if they're outdated and no longer supported with security patches. Similarly, in content management systems (CMS) like WordPress, insecure plugins can pose a significant risk if they have vulnerabilities that allow attackers to gain unauthorized access, inject malicious code, or perform other harmful actions on the website.

In applications like ChatGPT, which works on large language models, an insecure plugin is an external add-on or tool that integrates with the LLM but lacks robust security measures. An attacker could attempt to exploit the misconfigured or vulnerable plugins to compromise the data privacy and security of the victim users, introduce non-compliance with regulatory standards, and cause potential performance and stability issues.

About this lesson

During this lesson, we will learn how an attacker could attempt to utilize an insecure plugin to compromise the confidentiality and integrity of the victim user’s data. We will start by identifying the vulnerable plugin, understanding the vulnerable code and exploiting it to impact the application's users. We’ll then dive deep into the vulnerable code that allowed us to perform the attack and finish by fixing up the code.


Listening in

Did you know that in 2019, a popular voice-controlled smart speaker was found to have a vulnerability through a third-party app (akin to a plugin)? This vulnerability could potentially allow hackers to eavesdrop on conversations. While this didn't specifically involve a large language model, it highlights the real-world risks of insecure plugins or add-ons to advanced tech systems.

Insecure plugin in action

Attacking insecure plugins

  • STEP 1
  • STEP 2
  • STEP 3
  • STEP 4
  • STEP 5
  • STEP 6

Setting the stage

A new mental health AI tool has been released. It allows you to talk about anything on your mind and the conversations are confidential. Or are they?


Insecure plugin attack under the hood

Upon debugging the code, we found the following piece of code that was vulnerable:

We attempted to exploit the vulnerability by making a request with a malicious file to the POST /uploadProfilePicture HTTP/1.1 endpoint. A vulnerable request would look like the following:

POST /uploadProfilePicture HTTP/1.1
Host: mindtalk.ai
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="userID"
Content-Disposition: form-data; name="profilePicture"; filename="exploit.svg"
Content-Type: image/svg+xml
[malicious SVG content]

After sending the request, we observed that the server validated the file, and since it was not an accepted format, it triggered an error message. However, the error message was verbose to contain sensitive information like chat history, as shown in the response:

HTTP/1.1 500 Internal Server Error
Content-Type: text/plain; charset=utf-8
Connection: close
Error: Unsupported file type. User ID: user555, Chat History:
- User: "I've been feeling really anxious lately..."
- Bot: "I'm here to help. Can you tell me more about what's been going on?"
- User: "Work has been overwhelming, and it's affecting my sleep."
- Bot: "It sounds like you're under a lot of stress. Let's explore some techniques to manage anxiety and improve your sleep."

What is the impact of using an insecure plugin?

The impact of an insecure plugin is no less than using insecure or known vulnerable components in your application, be it a traditional web application or an LLM-based application.

The most immediate and serious consequence is the risk of data breaches. Insecure plugins can become gateways for unauthorized access to sensitive user data, as seen in our example above.

Organizations may face legal repercussions for failing to protect user data, especially if they are non-compliant with data protection regulations like GDPR, HIPAA, etc. Insecure plugins could expose proprietary algorithms, business strategies, or other intellectual property integral to the company’s competitive edge.

Insecure plugins can also impact the performance and stability of the overall system, leading to a poor user experience. Because of all of these reasons (and probably more!), we want to make sure our plugins are safe and we are only allowing third-party plugins that have been proven to be secure.

Scan your code & stay secure with Snyk - for FREE!

Did you know you can use Snyk for free to verify that your code
doesn't include this or other vulnerabilities?

Scan your code

Insecure plugins mitigation

Plugins should enforce strict parameterized input wherever possible and include type and range checks on inputs. When this is not possible, a second layer of typed calls should be introduced, parsing requests and applying validation and sanitization. When freeform input must be accepted because of application semantics, it should be carefully inspected to ensure that no potentially harmful methods are being called. This is true for all input!

Plugins should also use appropriate authentication identities, such as OAuth2, to apply effective authorization and access control. Additionally, API Keys should be used to provide context for custom authorization decisions which reflect the plugin route rather than the default interactive user.

The vulnerability in the code example above can be mitigated in the following way:

Let’s look at what changes were made in the code to make it more secure:

Validation checks: Added checks for the existence of userID and file to ensure that the request is valid.

Error handling: When an unsupported file type is uploaded, the server logs the error internally for review (without exposing sensitive information) and returns a generic error message to the user.

Response messages: The error messages returned to the client are generic and do not include sensitive data, such as user IDs or chat histories.

Status codes: Appropriate HTTP status codes are used to indicate the nature of the response (e.g., 400 for bad requests, 200 for successful uploads).

Keep learning

Learn more about insecure plugins and other LLM vulnerabilities.