Universal Broker

In this lesson, you will learn what the Universal Broker is and the principles by which it works. Snyk’s documentation for the Universal Broker contains command and script examples that are referred to in this lesson.

There are currently two versions of the Broker available:

• The Classic Broker, available prior to April 2025. If you are using the Classic Broker, and looking for information on how to use it, please see the Snyk documentation on the classic version of Broker.
• The Universal Broker, which this lesson covers, and is the planned future for Brokered communication. For more details, see Universal Broker in the Snyk documentation.

Requirements

• Snyk Enterprise plan customer.
• Tenant admin access to Snyk.
• Integrating a supported integration for Broker.
• See Snyk Universal Broker prerequisites for detailed system and software requirements

The Universal Broker is a communication mechanism that lets Snyk communicate with supported source code management systems (SCM), Jira, Container Registries, and package registries that are self-hosted in a private cloud or data center.

• The Broker client is a container deployed within your infrastructure where it can communicate with your platforms.
• It maintains a secure connection within Snyk.
• It allows credentials to remain behind the customer's perimeter.

You can think of the Universal Broker as plumbing or infrastructure. Once the Universal Broker is initially set up by the implementer(s), it allows developers, managers, or the security team to interact with Snyk seamlessly. This allows users to onboard projects from a brokered SCM and allows Snyk to be notified of pull requests in restricted environments, It also allows for creation of tickets in self-hosted Jira, or onboarding containers from self-hosted registries, within the access limits they have been given.

Advantages of the Universal Broker

• Connect to private or self-hosted platforms securely.
• No need for many Brokers. One Broker can connect to many platforms (Git, Jira, Container Registry, and more). Note that Snyk recommends no more than 25 connections per broker client (Deployment).
• Tokens are fully abstracted away! You are providing a reference to where they are stored but not actually sending or saving them in your broker configuration.
• The configurations, referred to as Deployments, are stored in Snyk's backend making it easy to configure and track.
  • Snyk does not store credentials, only pointers.
  • These pointers are credentials references, that is, environment variable names that Broker will reference in the customer environment.
• Streamlined connection management for Broker clients.

Differentiating the Classic Broker from the Universal Broker

If you have not used or read up on the Classic Broker, skip to the next section: "Universal Broker described".

If you have used the Classic Broker, you will see that the Universal Broker has been rearchitected and there are fundamental and very important differences from the Classic version:

• On startup, the Broker no longer gets connection configuration from the startup script or yaml file of the container.
• On startup, the container retrieves the configuration, known as a Deployment, from Snyk, and then spins up the connection(s) to the integrations specified in the Deployment. There may be many connections but the limit is 25 per deployment.
  • Prior to Universal Broker, the Classic Broker was designed around the concept of
    • one org=one integration=one broker
    • where any new Organization, integration or Broker would require a new broker. This is no longer the case! The Universal Broker can have many connections (integrations), and be associated with many Organizations
  • We will talk about this relationship more in the following sections.

Universal Broker described

Video 2m48s

Topics covered in this video:

• How does Snyk communication work over the web with publicly accessible platforms?
• How does Snyk communicate with self-hosted and private instances?

Before the Broker launches, it's important to note that an implementer or administrator will create a Deployment prior to launching/installing the Broker. A Deployment is essentially a configuration, saved on the Snyk platform, that contains:

• Configuration indicating what integration types to connect to.
• Connection information to those integrations and pointers to where to find the credentials (that is, environment variables)
• Associations for the Snyk Organizations and which connections they should have access to.

A Deployment can be created by installing snyk-broker-config CLI tool locally (recommended) or by using an API.

• For ease of use, Snyk recommends that users with the needed permissions create a Deployment by using the snyk-broker-config CLI tool, available for download from NPM.

After the Deployment is created, you can deploy the Broker client container in your environment for use.

Video 3m36s

Topics covered in this video:

• Deployments (high level)
• How Broker starts and creates connections

The following sections outline how each integration behaves. Snyk recommends that you review the videos related to the integrations you plan to implement, as each integration comes with different configuration options and considerations. For example:

• SCM: You may configure code repository webhooks for pull request checks differently based on firewall and infrastructure limitations.
• Container registry: You will configure and use Snyk Broker Container Registry Agent alongside the Broker

Video 4m 38s

It's important to note that the webhook URL mentioned is set by the broker_client_url value set during the connection setup stage. Normally it's defaulted and recommended to be api.snyk.io (broker_client_url=https://api.snyk.io or your region's equivalent). Otherwise, you would point to the URL of the running Universal Broker container running in your environment

Topics covered in this video:

• Importing a code repository from source code management (SCM) into Snyk.
• How Snyk Pull Request checks work?
  • SCM Import through Broker setting the webhook to direct connection with api.snyk.io
  • SCM Import through Broker setting a webhook to the broker client behind the customer's infrastructure.
• How does Snyk create pull requests to fix security issues in open source manifests and dockerfiles?

Video 0m51s

Topics covered in this video:

• Creating a Jira ticket from within Snyk

Video 1m01s

Topics covered in this video:

• Importing a container from a container regisry
• Snyk Broker Container Registry Agent

If you are not using Classic Broker, you can skip this step and move on to "Creating a Deployment", otherwise, for customers using the Classic Broker, it's important to note that after creating the connection, you will be required to run a migrate related workflow. For more information, it's recommended to read the following article prior to proceeding to the following steps where you learn to create a connection, and integrate it.

Article: Upgrade an Organization integration from Classic Broker to Universal Broker

Requirements

• You must be a tenant admin to create deployments, credentials reference(s), and connection(s).
• For additional requirements, see the prerequisites in the user documentation.

Universal Broker data model

Deployment creation methods

There are two methods to create Deployments:

• Install snyk-broker-config, a CLI tool, locally
  • Users with the needed permissions create a Deployment using the snyk-broker-config cli tool
• API
  • A user creates the deployment over several API calls.
  • Using the snyk-broker-config CLI tool is recommended for an easier experience. The API allows for automation and more control, and also requires a clear understanding of Broker deployments, credentials, and connections.

After you create Deployment(s), you can deploy and use the Broker client in your environment.

For a summary of the steps in creating a deployment, with connections, and integrations, see Basic steps to install and configure Universal Broker in the user documentation.

Creating a Deployment

• Install or update the snyk-broker-config application locally.
• You will be prompted for a Snyk Account token. This token will require an API token for a user with Tenant administrative privileges.
• You will start the Universal Broker Create Connection workflow to create your first connection
  • One time task: As part of this step you will be prompted to install the Snyk Broker App in any Organization in your tenant. We suggest you use the Broker Admin Organization referred to in the Prerequisites
    • You have to install the Snyk App only once, regardless of the number of Organizations, where you want to use Universal Broker and the number of connections you wish to make.
    • During later workflows, you may get prompted with a check to see if you have the Snyk App installed. You can indicate "Yes", and continue past the prompt without installing if it is the same tenant you had previously installed the Snyk App on.
  • This returns an install ID, a client ID, and a client secret; all are needed to interact with the Snyk platform. The client ID and client secret should be safely stored as it gets generated once and will be needed later.
• Create a Universal Broker deployment for your tenant ID and install ID. The Organization ID is required to create the deployment.
• Create credential references needed for your connections
• Create your desired connection or connections.
• One time task for each Organization integration: Configure the Organizations and integrations that should use the connection using the integrate workflow.

The following diagram illustrates Deployment creation:

The following diagram illustrates creating the Connection:

The following diagram illustrates the Connection and Organization integration process:

Refer to Snyk Universal Broker prerequisites documentation

Refer to Basic Steps to install and configure Universal Broker for a walkthrough of commands for setting up deployment with snyk-broker-config.

• You will need
  • Permissions to configure Snyk’s Universal broker container in your environment.
• To Run your Broker deployment on your container engine or Kubernetes cluster
  • The broker launch script contains the client ID, and secret retrieved from Snyk. These values were returned as a result of the Broker App application at the start of this workflow.
    • Broker uses OAUTH to startup
    • You will need the Deployment ID from the initial steps.
  • Target your desired environment with -e BROKER_SERVER_URL=https://broker.REGION.snyk.io \ if you are not using broker.snyk.io (where REGION contains the correct region value).
  • Add the environment variable or variables as defined in the credentials references with the associated value. The connection will not be established if references are missing, and an error entry will be logged in the Broker Client logs.

Refer to Snyk Documentation for official script options.

This section provides a list of considerations for each major integration type for the Universal Broker.

Please refer to the provided documentation links in each description for more information.

Org level Git/SCM Broker considerations

• Configure your deployment connections.
  • Will the pull request use the default for the webhook, going directly to Snyk? You define this while creating a connection setting the broker_client_url
  • Or do you need to set the SCM webhook to the Broker client?
• Configure integration in Snyk for that integration in your organization(s), for behaviors like how PR checks will behave or automatic fix PR behaviors.

Org level Jira Broker considerations

• Configure your deployment connections.
• Configure integration settings in Snyk for Jira, choosing the broker setting.
• Configure your Jira settings.

Org level container registry considerations

For container registries where Universal Broker is required, and you do not want to send the full container to Snyk, the following are required:

• Configure the Snyk Container Registry Agent to run alongside broker. This allows dependencies to be processed locally in the customer's environment, followed by sending the dependency tree to Snyk, instead of the whole container.
• Configure container registry integration in Snyk.
• For the broker_client_url value set during connection workflow, Snyk requires the broker client address. Dns hostname is recommended (http://my.broker.client) but IP addresses can also work (http://192.168.0.1). The port should be appended (for example http://my.broker.client:8000).
  • Note that https webhook calls require additional setup in the broker client for you to bring a TLS certificate and mount it into the container.

Org level package Registry considerations

Snyk can connect to Artifactory and Nexus through the Universal Broker for dependency resolution and Fix PR generation.

Artifactory

• Create and configure an Artifactory Package repository connection, using the Broker option
• Add the relevant Maven or npm settings

Nexus

• Create and configure a Nexus Package repository connection, using the Broker option
• Add the relevant Maven or npm settings

Snyk Essentials and Snyk AppRisk

In order to create a Group-level connection for Snyk Essentials and Snyk AppRisk to your SCMs, the following steps are required:

• Universal Broker connections are usually associated at the Snyk Organization level. To accommodate this, you are going to create a reference organization, with all the SCM integration connections you want to use within Snyk Essentials and Snyk AppRisk. These integrations should use the permissions and have broad visibility across your code repositories so that the Universal Broker can compare what you have versus what security controls are applied, such as SAST.
• After the reference Organization is created, each of the integrations have Broker Ids, also referred to as “Broker tokens” in Classic Broker, that you will retrieve from each integration settings screen, and associate with your group-level integration. This will allow each group-level integration to use the Universal Broker connections you setup in your reference Organization.