• Browse topics
Login

Unapproved change

You did not see that coming

General

Unapproved change: the basics

What is unapproved change?

Unapproved change refers to situations where an open source component changes in an unintended or unnoticed way after it is integrated into a project. This risk often arises when developers rely on mutable resources, such as non-versioned scripts, unverified repository references, or components delivered over insecure protocols. These changes can occur due to tampering, upstream modifications, or an unintentional oversight in the distribution process.

Such changes can lead to vulnerabilities, including supply chain attacks like the Codecov Bash Uploader breach, in which a malicious actor inserted a backdoor into a widely used tool. Developers may unknowingly propagate these vulnerabilities throughout their software systems, compromising security and stability.

About this lesson

In this lesson, you will learn how unapproved changes occur in open source components and why they pose a significant security risk. We will examine a real-world inspired scenario to demonstrate how mutable components can lead to severe vulnerabilities. You will also explore methods to prevent and detect such risks in your projects.

FUN FACT

Codecov Bash Uploader

Did you know that a 2021 incident involving the Codecov Bash Uploader exploited mutable components to inject malicious code into thousands of repositories? The breach occurred because the upload script was downloaded from a non-versioned URL, allowing attackers to replace the legitimate script with a malicious one. This attack remained undetected for months and impacted major organizations, highlighting the dangers of relying on mutable or non-validated components in CI/CD pipelines. You can learn more about it here.

Unapproved change in action

Robin is a backend developer at TechNovaXYZ, a company that prides itself on shipping secure, reliable software. While working on a high-performance analytics feature, Robin integrates an open source library, fast-compute, from a well-known GitHub repository.

In the requirements.txt file, Robin adds:

git+http://github.com/trusted/fast-compute

No specific tag or commit hash, just the latest from the default branch.

Months later, support tickets begin piling up. Users report performance issues, corrupted analytics reports, and eventually, one user notices that sensitive data appears to have leaked.

Robin begins investigating. First stop: check what commit of fast-compute was pulled.

Demo terminal

That doesn’t look familiar. Robin digs into the commit history and compares the current code with the previously tested version.

git diff 6a2f3bc9a8d7 e1f4d9a98b3c

Robin’s heart sinks. The fast-compute library has been compromised. Malicious code has been quietly added to steal sensitive data and it slipped through because the team never pinned the dependency or verified upstream changes.

Unapproved change under the hood

Let’s break down how Robin’s situation unfolded, revealing the technical risks and lessons from the example.

Unversioned dependencies

The requirements.txt entry git+http://github.com/trusted/fast-compute points to the latest version of the repository's default branch. While this simplifies dependency management, it introduces the risk of unintended updates. Any change to the repository, whether legitimate or malicious, affects all systems relying on this reference.

Lack of integrity verification

Without mechanisms like checksums or digital signatures, there’s no way to verify that the downloaded library is the intended artifact. In Robin’s case, the modified fast-compute library included malicious code that exfiltrated sensitive data, but there was no validation process in place to detect the tampering.

Tampered code execution

The CI/CD pipeline automatically fetched the latest changes, incorporating the tampered library into production. The malicious code is executed at runtime, exploiting the trust placed in the dependency and causing a breach in the application’s security.

Secure practices

To prevent such risks, developers should use immutable references, such as specific commit hashes or version tags, to lock dependencies and ensure that the same version of a component is always used.

Additionally, developers should verify the integrity of downloaded components by utilizing cryptographic signatures or hash values, which can detect tampering or unauthorized modifications. Regularly auditing dependencies and reviewing upstream repositories for suspicious changes or unauthorized access further enhances security. By implementing these practices, teams can better maintain the stability and security of their software, even when relying on external components.

Vulnerable process example

Imagine a CI/CD pipeline set up for a software project. The project relies on several open source libraries to handle tasks like authentication, logging, and data processing. To save time, the developers decide to pull the latest versions of these dependencies directly from their respective repositories.

In the pipeline configuration, a dependency is included like this:

dependencies:
- source: http://github.com/trusted/fast-compute
type: git
branch: main

The pipeline is designed to automatically fetch and build the latest code from the main branch of the fast-compute repository during every deployment.

Over time, the maintainers of fast-compute step away from the project, leaving it unmaintained. An attacker notices this and gains unauthorized access to the repository. They inject malicious code that exfiltrates sensitive data, such as database credentials or API keys, whenever the library is initialized.

When the pipeline runs again, it fetches the updated library without any checks for integrity or authentication of the source. The malicious code is executed as part of the application’s deployment process, compromising the security of the entire system.

What is the impact of unapproved change?

When a dependency is modified without the knowledge or approval of the developers, it can introduce malicious code, security vulnerabilities, or unintended behavior into the application. This could lead to data breaches, unauthorized access, or exfiltration of sensitive information, as demonstrated in high-profile incidents like the Codecov Bash Uploader breach. Another significant impact is the disruption of software stability and reproducibility. Mutable dependencies mean that the same codebase could behave differently across builds, leading to unexpected bugs and downtime. For businesses, this can translate to financial losses, damaged reputation, and compliance failures, especially when software does not meet security and operational standards.

Finally, unapproved changes undermine the reliability of supply chains. Dependency tampering can cascade through the ecosystem, affecting not just the direct users of a library but also any downstream systems. In complex software environments where dependencies rely on other dependencies, this can create widespread vulnerabilities that are difficult to track and mitigate. By addressing these risks proactively, organizations can safeguard their software against the potentially catastrophic effects of unapproved changes.

Scan your code & stay secure with Snyk - for FREE!

Did you know you can use Snyk for free to verify that your code
doesn't include this or other vulnerabilities?

Scan your code

Unapproved change mitigation

Mitigating the risk of unapproved changes requires a combination of technical controls, process improvements, and best practices. These measures focus on ensuring the immutability and authenticity of third-party components and their secure integration into your software. Use immutable references

Always reference specific versions or commit hashes for dependencies. For example, instead of pointing to the main branch of a repository, specify a stable version tag (v1.2.3) or a unique commit hash (abcdef123456). This ensures that every build uses the exact same version of a dependency.

Verify component integrity

Use cryptographic signatures or checksum verification to validate that the downloaded components are untampered. Many package registries, like PyPI, Maven, and npm, support this functionality. For custom integrations, maintain a list of precomputed hashes for approved dependencies and check the downloaded artifacts against these values.

Prefer secure protocols

Always use secure protocols, such as HTTPS or SSH, for downloading dependencies. These protocols encrypt data in transit and help prevent man-in-the-middle (MITM) attacks. Avoid using unsecured protocols like HTTP for fetching components.

Audit and monitor dependencies

Conduct regular reviews of your dependencies. Audit their repositories for unexpected changes or suspicious commits. Use tools like Snyk to monitor for vulnerabilities and updates in your third-party components.

Implement a dependency firewall

Use tools or policies that restrict what dependencies can be added to your project. For example, a dependency firewall can block additions that are not explicitly approved, originate from untrusted sources, or lack integrity checks.

Enforce security in CI/CD pipelines

Configure your pipelines to fail if dependencies are not immutable, lack proper signatures, or do not match expected checksums. Integrate tools that automatically check dependency security during the build process.

Mitigated process example

Let’s revisit the CI/CD pipeline scenario, now incorporating the mitigations to secure the integration of the fast-compute dependency. The updated pipeline configuration specifies an immutable version and verifies the integrity of the downloaded dependency: dependencies:

- source: http://github.com/trusted/fast-compute
type: git
commit: 3f4a8c2d9b6e7a9f1d1b3c0e1f2d8a7e9f3b0c1d
checksum: sha256:47f3060e62314b4d2bc5e0da4f2c8e4dabe5e3c9e1b2f1d4a7f3c2d0a1e6d9b3

The configuration specifies an immutable version by referencing a specific commit hash. This ensures that every build fetches the exact same version of the code, preventing accidental updates or tampering from affecting the pipeline. Additionally, the pipeline includes integrity verification by specifying a precomputed hash of the dependency. When the artifact is downloaded, the pipeline calculates its checksum and compares it to the expected value. If there is a mismatch, the build fails, blocking any unverified or malicious changes from being deployed.

The use of secure protocols is another improvement in the updated process. The original http protocol has been replaced with https, which encrypts data in transit and protects against man-in-the-middle (MITM) attacks. Furthermore, the team actively monitors the dependency using automated tools that scan for vulnerabilities and alert them to changes in the upstream repository. This proactive monitoring ensures that the library remains safe and aligns with security policies.

This is how the dependency is securely included into the codebase.

Fetch the Dependency

The pipeline clones the repository using the specified commit hash:

git clone http://github.com/trusted/fast-compute
cd fast-compute
git checkout 3f4a8c2d9b6e7a9f1d1b3c0e1f2d8a7e9f3b0c1d

Verify integrity

After downloading, the pipeline calculates the SHA-256 checksum of the dependency: sha256sum fast-compute.tar.gz

If the computed checksum doesn’t match the expected value, the build process stops, preventing the deployment of unverified code. Once verified, the dependency is integrated into the build process, and the system ensures that no additional components are fetched dynamically without validation.

Quiz

Test your knowledge!

Quiz

What is an effective way to reduce the risk of unapproved changes in open source software?

Keep learning

To deepen your understanding of managing open source dependencies and protecting against risks like unapproved changes, consider exploring the following resources:

Congratulations

You’ve taken your first step into understanding what unapproved changes are, how they work, the potential impacts they can have, and how to mitigate these risks effectively. By applying this knowledge, you can ensure your applications are more stable, secure, and resilient to threats arising from mutable components.