Insecure plugins for LLMs

An insecure plugin refers to a software component or extension that contains vulnerabilities or weaknesses that could potentially be exploited by attackers to compromise the security of a system or application. These plugins are often used to add functionality or features to a larger software platform or system, but if they're poorly designed or implemented, they can introduce security risks.

In the context of web browsers, plugins or browser extensions can be examples of insecure plugins if they have known security flaws or if they're outdated and no longer supported with security patches. Similarly, in content management systems (CMS) like WordPress, insecure plugins can pose a significant risk if they have vulnerabilities that allow attackers to gain unauthorized access, inject malicious code, or perform other harmful actions on the website.

In applications like ChatGPT, which works on large language models, an insecure plugin is an external add-on or tool that integrates with the LLM but lacks robust security measures. An attacker could attempt to exploit the misconfigured or vulnerable plugins to compromise the data privacy and security of the victim users, introduce non-compliance with regulatory standards, and cause potential performance and stability issues.

During this lesson, we will learn how an attacker could attempt to utilize an insecure plugin to compromise the confidentiality and integrity of the victim user’s data. We will start by identifying the vulnerable plugin, understanding the vulnerable code and exploiting it to impact the application's users. We’ll then dive deep into the vulnerable code that allowed us to perform the attack and finish by fixing up the code.

Upon debugging the code, we found the following piece of code that was vulnerable:

We attempted to exploit the vulnerability by making a request with a malicious file to the POST /uploadProfilePicture HTTP/1.1 endpoint. A vulnerable request would look like the following:

After sending the request, we observed that the server validated the file, and since it was not an accepted format, it triggered an error message. However, the error message was verbose to contain sensitive information like chat history, as shown in the response:

The impact of an insecure plugin is no less than using insecure or known vulnerable components in your application, be it a traditional web application or an LLM-based application.

The most immediate and serious consequence is the risk of data breaches. Insecure plugins can become gateways for unauthorized access to sensitive user data, as seen in our example above.

Organizations may face legal repercussions for failing to protect user data, especially if they are non-compliant with data protection regulations like GDPR, HIPAA, etc. Insecure plugins could expose proprietary algorithms, business strategies, or other intellectual property integral to the company’s competitive edge.

Insecure plugins can also impact the performance and stability of the overall system, leading to a poor user experience. Because of all of these reasons (and probably more!), we want to make sure our plugins are safe and we are only allowing third-party plugins that have been proven to be secure.

Plugins should enforce strict parameterized input wherever possible and include type and range checks on inputs. When this is not possible, a second layer of typed calls should be introduced, parsing requests and applying validation and sanitization. When freeform input must be accepted because of application semantics, it should be carefully inspected to ensure that no potentially harmful methods are being called. This is true for all input!

Plugins should also use appropriate authentication identities, such as OAuth2, to apply effective authorization and access control. Additionally, API Keys should be used to provide context for custom authorization decisions which reflect the plugin route rather than the default interactive user.

The vulnerability in the code example above can be mitigated in the following way:

Let’s look at what changes were made in the code to make it more secure:

Validation checks: Added checks for the existence of userID and file to ensure that the request is valid.

Error handling: When an unsupported file type is uploaded, the server logs the error internally for review (without exposing sensitive information) and returns a generic error message to the user.

Response messages: The error messages returned to the client are generic and do not include sensitive data, such as user IDs or chat histories.

Status codes: Appropriate HTTP status codes are used to indicate the nature of the response (e.g., 400 for bad requests, 200 for successful uploads).

Learn more about insecure plugins and other LLM vulnerabilities.

• OWASP slides about their new LLM top 10 list
• A dedicated OWASP page to LLM07, LLM07: Insecure plugin design
• An insecure plugin attack is defined with a CWE as well. More about this can be found at the official MITRE website.
• A Snyk publication on the OWASP LLM top 10 list