Open-Source Component Best Practices
If you use open-source components in your apps, that means you have risk. Even the most cautious consumers are vulnerable. We are in the business of keeping your software supply chain protected and want to share with you our own open-source best practices.
You've Got Risk.
Let’s face the facts: open-source components are risky. 29% of popular open-source components contain a known vulnerability. 2021 saw a 650% increase in the number of cyber-attacks aimed at open-source suppliers. And events like the log4j vulnerability in late 2021 proves that even pervasive, actively supported components can leave consumers wide open to attack.
And yet, 2021 saw a 73% growth in open-source component downloads. Why is open-source exploding even though the risks are high?
It’s because the benefits of open-source components outweigh the risks. Apps that contain open-source components are developed and released faster, and are higher quality and more stable. It’s not just a nifty shortcut or an occasional luxury. Open-source components are part-and-parcel with how software gets made. And that means that your applications are going to have some open-source component risk.
In this content, we will discuss best practices for managing that risk.
Pressed for time? TL;DR
- Baseline your existing component usage.
As part of that, you need to understand the different kinds of risk that components can have. You also need to get comfortable with the idea of transitive dependencies.
- Determine an acceptable level of risk for your organization and your apps.
- Create a governance policy to communicate your risk tolerance and establish a process for dealing with it.
- Whenever possible, mitigate or remediate risks in a flexible, but repeatable, way.
A Balancing Act: Risks and Benefits of Open Source
It’s important to understand that not all of the risks presented by open-source components are necessarily deal-breakers. Organizations must weigh the pros and cons of each risk to know what is acceptable or unacceptable for their company. Adopting the best practices outlined in this guide begins with looking at all aspects of open source software, which will help inform the approach your organization takes.
The Benefits
Quickly adapting to change is a critically important competitive advantage. Nowhere is this more apparent than how open source is accelerating innovation in software development. With judicious open-source component usage:
- New projects are prototyped and deployed in a few days.
- Cutting-edge technologies can be integrated with just a few lines of code.
- Common architecture makes it easy to bring skilled developers quickly to the team.
- The costly process of vendor acquisitions are partially or completely avoided.
- Development costs are spread across the entire community.
The Risks
On the other hand, the risks of open source are not as clear, sometimes not emerging until later in an application’s lifecycle. The relative risk of an application’s open source dependencies increases as those components age. Such risks can include:
- May have widely known security vulnerabilities that are easy targets for hackers.
- Legal troubles about open-source licenses and the high cost of complying with unexpected restrictions.
- Can be costly to upgrade when they have accumulated breaking changes over multiple versions.
- Issues go unresolved as contributors move to other projects.
- ‘Stable’ components can cease being under active development.
Any of these will eventually result in a large amount of tech debt that is challenging for development teams to prioritize over new functionality. This becomes worse for legacy applications that no longer have active development budgets.
Your Two-Sided Relationship with Components
Your relationship with components has two stages – pre-consumption and post-consumption. Focusing on post-consumption risk before pre-consumption risk, however, will make the best use of your time.
Pre- and Post-Consumption
There are two stages in your relationship to open-source components. The first stage is pre-consumption, when you “go shopping” for a component that satisfies a specific need in your app. The second stage is post-consumption, when the component is incorporated into the app and appears in a build.
Broadly speaking, pre-consumption best practices are about comparing a component’s usefulness to its quality and level of risk. Post-consumption best practices are about mitigating existing risk and weighing the costs and benefits of your remediation options.
Your instinct might be to say that apps are often in both stages at once. That’s true; the breakneck pace of modern software development and the emphasis on CI/CD means that apps are usually in both stages for a generous portion of their lifetime.
Managing your risk effectively means addressing both stages. Focusing on the pre-consumption stage exclusively is a bad strategy because there is no such thing as a risk-free component. Nothing can totally prevent component risk. Conversely, focusing on the post-consumption stage exclusively is also a bad strategy because the quickest, most scalable way to reduce your risk is to select better components from the start. Post-consumption strategies always have a higher cost than their pre-consumption counterparts.
Start With Post-Consumption Best Practices
This content is focused on providing best practices for components that you’ve already consumed. That’s because, if you’re just getting started, post-consumption best practices are the most effective use of your time.
Sometimes, the only correct pre-consumption strategy is to accept the risk, use the component, and commit to remediating the risk later. The remediation process is something you’ll codify when discussing post-consumption best practices, so it makes sense to focus your efforts there first.
Additionally, in the same way that your teeth need to be brushed every morning and night, you’ll always need to think about the open-source component risk in your applications. It’s inevitable; components are just too useful not to use! And no amount of caution, skill, tooling, or hard work can prevent components from bringing risk into your app.
Best Practice #1
Baseline Your Existing Component Usage
Use SBOMs to identify all the components in your application, and employ SCA tools to compare those components to known lists of vulnerabilities.
You’re probably reading this guide to find out how to remove some, if not all, of the risk from your product. Specifically risk in your software application that comes from open source software components. But your software, even software built from scratch, will have some risk. Problems from open source components like security vulnerabilities, license restrictions, and quality issues can be troublesome to read about, especially with high profile breaches like SolarWinds and Equifax. But the benefits of OSS far outweigh the risks when you manage your software correctly.
Open source packages save your developers tremendous amounts of time by letting them focus on solving the challenges unique to your application and can provide thoughtful, thoroughly tested solutions to complex problems for free.
Establishing a Baseline and Finding your Risk Tolerance
Since risk is inevitable, the solution isn’t to try and eliminate risk, but to manage it. To do that, you will need to establish a baseline of what risk already exists. Once you know that benchmark, you can define a set of rules defining what risk is acceptable and what isn’t. This is your organization’s risk tolerance. See more on that in the next section.
A common pattern when teams adopt scanning tools is to scan an application, then try to remove all the vulnerabilities their Software Composition Analysis (SCA) tool finds.
Here’s how to approach establishing your risk baseline:
Identify the components in your application.
The best way to do this is to generate a software bill of materials or SBOM. An SBOM lists every component in your application and their relationship to other components in a standard format that can be read easily by both humans and computers. The two SBOM standards are CycloneDX and SDPX. Our recommendation is CyclconeDX, since it’s developed specifically for monitoring security vulnerabilities.
The software bill of materials is the first step in managing your risk from open source packages since it gives you a way to identify the applications you’re actually using, including packages brought in from your dependencies. This report can be used with software composition analysis tools to generate a report on your vulnerability information, shared with customers for assessment, and used to monitor your application for new dependencies.
Eventually, you should create a vulnerability exploitability exchange and connect it with your software bill of materials. This companion document lets you track security vulnerabilities in your application and indicate when possible vulnerabilities do not apply to your implementation of the package.
Identify the risk from your existing components.
Once you know what you’re using, the next step is to identify your existing risk. The best way to do this is by using a software composition analysis tool. SCA tools can automatically determine which components you’re using and then check their available vulnerability databases to give you detailed information on the potential risks from your dependencies. The best tools will check your project’s build artifacts against public and proprietary information sources to determine potential risks, while others will check the application manifest file or SBOM against publicly available data. Some common SCA tools include Nexus Lifecycle, OSS Index, and OWASP Dependency Check.
Types of Risk
It’s no secret that open-source components can be security risks, but that risk takes many forms, and not every risk is equally severe. In this guide, we review the concept of security risk and give you a few different ways of judging severity.
Read more…
Open-source license risk includes both the cost of complying with a license and the cost of failing to comply. This guide discusses both costs and overviews how to make sense of the open-source licenses you’ll find “in the wild.”
Read more…
Sometimes you have to judge a book by its cover if it means having a safe application rather than one riddled with vulnerabilities. Knowing how to spot components with high quality risk keeps you vigilant and your application robust.
Read more…
Open source components typically are not operational on their own. Transitive dependencies provide auxiliary functionality for other dependencies to work.
Read more…
Decide if anything needs to be remediated.
Deciding what to remediate when you start managing your open source risk is tricky. Rarely do you want to go back and remove all of your existing risk, especially in older applications and large codebases. Remediation takes time away from developing your applications. Redoing existing work can frustrate your development teams and take attention away from the processes of proactively managing risk moving forward.
A common approach is to accept all but the most severe risks. Any extreme vulnerabilities, like easy-to-exploit, remote code execution, should be addressed as these represent a huge danger to your organization and potentially your users.
Moving Forward
The goal with open source risk management is to innovate as fast as possible without taking unnecessary risks. Oftentimes, your existing applications won’t completely meet the open source governance policies your organization has adopted. This is a compromise, but it allows your company’s focus to be on newly emerging, active risk rather than removing existing risk that might simultaneously create more risk. This proactive approach will ensure that your future projects will be less risky through proactive risk mitigation.
Best Practice #2
Set Your Risk Tolerance & Prioritizing Risk
Set your risk tolerance on a per-app basis, and consider the risk’s severity along with the the cost of remediation. Budget for tech debt and divide your apps into tiers based on their importance to ensure the budget is distributed effectively.
Understanding Risk Tolerance
It’s important to acknowledge that your applications will never be totally free of component risk. In fact, if you want an application free of component risk, you’ll need make an application without any components, and committing to that development budget probably isn’t feasible for your organization.
Instead, you’ll need to decide on a level of risk that you’re willing to accept — meaning, the risks that you won’t try to remove, mitigate, or remediate. This is your risk tolerance. Some risks are small enough that they can be safely ignored. Other risks are more serious, but don’t make sense to put any development budget towards. And some risks are severe but simply can’t be addressed.
After that, you’ll need to prioritize addressing the remaining risk. Some issues will need fixed immediately, and others can wait until your next sprint cycle, your next big release, or even your next round of funding.
Setting Tolerance on a Per-App Basis
For a given application, there are a number of factors to consider when estimating the acceptable risk level, such as the following:
- What is the risk’s relative level of threat to the organization?
- Can the component be upgraded to a less-risky version?
- Will upgrading the component require rework to fix breaking changes?
- Is the vulnerability exploitable in the application? Can the exploit be mitigated?
- Could another open source component be used instead?
Budgeting for Tech Debt
Compliance with any free and open source software (FOSS) governance policy will slow innovation as a factor of technical debt. This equates to real development hours in dependency upgrading, CI testing, issue remediation, and, unfortunately, rework. A project will commonly allocate between 10-30% of an application’s budget to manage technical debt. Determining an application’s acceptable level of risk helps to prioritize remediation to fit these constraints.
To handle tech debt effectively, group applications in tiers depending on how critical they are to business continuity and impact, then assign tech debt budget accordingly. The following questions can help determine which tier of risk the application belongs to:
- Does the application provide critical competitive advantage for the organization?
- Does the application handle payments, orders, or personal information?
- What is the cost of the application leaking user data?
- Is the application internal- or external-facing?
- Is the application in active development or legacy code?
- Are there any external compliance requirements for the application?
- Will an issue introduce risk into other applications?
Don’t Solve the Issue of Aging by Automatically Updating
Some projects avoid aging issues by automatically upgrading their dependencies when new versions are available. This is often accomplished by including a version range in the manifest files, either without an upper bound or one set to the next major version. This practice introduces another set of issues that is rapidly becoming riskier than aging. In general, we always recommend either pegging dependency versions and/or using project lock files to use versions that have been first vetted. Risks with open versioning include:
- Trusting communities are widely exploited from contributors with malicious intent.
- Dependency confusion attacks that target unreserved namespaces are now very common.
- New releases may inadvertently introduce breaking changes.
- Many available SCA tools perform namespace-matching without verifying the binaries used.
Best Practice #3
Create and Maintain a Governance Policy
Develop a governance policy using the Crawl/Walk/Run framework. Open feedback channels, watch for bottlenecks, and automate for speed and predictability.
Sitting Down And Getting Started
The governance policy is an important tool in your component risk management toolbox. It serves as a public statement about your risk tolerance and a set of guidelines that others will use to guide decision-making. The best governance policies are short, plain-language, and available to everyone in your organization.
If you don’t have a governance policy already, start small and be focused. Your first governance policy should be limited in scope, make small and calculated asks, and include just key stakeholders. As muscle memory is built and processes are refined, slowly build out to include more teams and bigger asks. It’s not wise to try and implement a mature, nuanced governance policy right off the bat – you’ll burn through resources and goodwill almost immediately. For that reason, if you’re just getting started, consider the crawl/walk/run model.
As your governance policy comes online, watch for bottlenecks, and open feedback channels so that others can alert you when they arise. These bottlenecks will first appear where Development and Operations meet Security. Solving these bottlenecks is a matter of scalability and repeatability, and that means automation.
Crawl – SBOMs and Risk Assessment
During your first pass at a governance policy, focus on getting SBOMs created for all new and existing apps. Assign accountability to your Development teams to generate and store an SBOM for every deployment build of their app. These SBOMs serve as point-in-time records of your apps, and legal requirements around SBOMs are on the horizon, so plan to keep these for 10 years or more.
Additionally, look to perform risk assessments on new and existing apps. Likely your AppSec team is already accountable for this. Store these along with your SBOMs and make a point to review them regularly at cross-functional team meetings. Even if you’re not remediating issues, being aware of risks is a huge step in the right direction. If something does go wrong – and it will, inevitably – having risk assessments on-hand reduces your response time.
There are automated tools that can help keep this low-effort and scalable. Sonatype Lifecycle is one such option. The CycloneDX project has a number of open-source tools that can help to keep your initial costs low.
Walk – Remediate
Once your organization is comfortable with Crawling, update your governance policy to address the topic of remediation. At the micro level, assign accountability for the act of remediation, and enable those individuals to make component risk management a priority. For example, give them the authority to create and move tickets in your issue tracking software (Jira, Assembla, etc.) specifically for remediation work.
Also at the micro level, establish the ideal remediation path. Usually, the best way to remediate is to update the component to a non-vulnerable version. If that’s not possible, describe what other remediation paths should be attempted. If remediation isn’t possible, lay out how mitigation efforts should be documented, or how risk can be temporarily waived until a later date.
Run – Establish Goals and Enforce
Once the remediation process is familiar, you can start to Run. In this stage, update your governance policy to establish what your organization’s goals are vis a vis component risk management. This goal will evolve as your organization evolves. As a result, your goals should be revisited often and have some flexibility built-in.
For example, you could start by declaring that your apps can’t be deployed if they contain component security risks over a certain severity level. Later, you might update to include a rule about component licenses, as well.
This stage is also where you’ll start enforcing your goals by actively blocking builds, deployments, and downloads of packages with unacceptable levels of risk. Enforcement is always a tricky subject – we’ll talk more about it shortly – but you can set yourself up for success by automating this enforcement. Ideally, blocking should happen automatically, based on a numerical score generated by a tool, with feedback piped directly to the people who are involved. Don’t rely on the instincts of someone from your AppSec team; that’s prone to human error and will only lead to frustration.
Acknowledge the Tension Between Rules and Innovation
There’s always a tension between making rules and encouraging innovation. This is especially true when it comes to enforcing your governance policy. You can alleviate that tension in a few ways.
First, make it clear that managing component risk is a company-wide effort, and not the “fault” of any individual or team. For example, a developer who picks a risky component hasn’t failed; component risk management isn’t so simple. You’ll notice that we like to use the word “accountability” rather than “responsibility” for this reason. Assigning responsibility is like assigning blame, whereas accountability is like asking for an explanation.
Second, avoid the scan-and-scold mentality. It’s not sustainable, and pitting AppSec against your developers guarantees slow, complex, and frustrating deployments.
Third, be crystal clear about what, when, and how things will be blocked, and lay out the path for getting unblocked. Handle blocking and unblocking in an automated way, for both speed and predictability.
Finally, acknowledge the tension in plain language! A governance policy isn’t about shooting down new ideas or slowing development to a crawl; it’s about ensuring that your apps are high-quality, stable, and innovative. Say that much, and create a place for feedback. Like any DevOps effort, your governance policy is a collaboration.
And, as you might expect, socializing your governance policy will be key.
Best Practice #4
Have Mitigation and Remediation Plans
Differentiate between Remediation and Mitigation. Upgrade to new, non-vulnerable versions of your components whenever possible, and waive risks that you can’t or won’t tackle.
Organizations known for successful vulnerability management have proactively created both mitigation and remediation plans to handle threats and vulnerabilities when they arise. While it is cost-prohibitive to remove risk entirely, these two approaches to risk management play distinct and very important roles in protecting organizations’ applications from third-party threats. They may sound similar or even interchangeable, but their differences are important.
- Remediation means correcting or counteracting something. This is the more permanent approach in a vulnerability management response. Remediation aims to abate the threat entirely. A remediation plan is more than simply having a flowchart or decision tree to guide an organization’s response to a vulnerability. There are numerous factors to consider, and there is no perfect equation to adequately remediate risk because each organization has unique business priorities and differing thresholds for risk.
- Mitigation, on the other hand, means making something less severe. In an ideal world, your applications are free from vulnerabilities because your organization’s remediation plans effectively prevented them. When this isn’t the case, however, the alternative is accepting the risk, which is at the heart of mitigation. An organization knows the risk is there but but determines that it is acceptable risk when weighed against the effort/cost to address it through remediation.
Remediation Strategies at a Glance
There are typically four general strategies to remediate vulnerabilities, each with their own pros and cons based on an organization’s specific set of circumstances. They are as follows:
| Strategy | What it is | Pros | Cons | Cost/Effort | Risk |
|---|---|---|---|---|---|
| Component Upgrade | Upgrade to better version | • Most effective • Least expensive • Can be automated |
• Wait time for version releases • Transitive dependencies problematic • Requires slower pace to prevent introducing more risk • Upgrading legacy code and breaking changes cost-prohibitive |
Low-Medium | Low |
| DIY Patch | Remove vulnerable code from execution path. Patch the component yourself; push back to maintainer’s repo. | • Opportunity to contribute to OSS community • A cheap and quick solution |
• Creates technical debt • Long-term commitment to maintenance • Difficult to remove all risks |
Medium-High | Medium |
| Switch Components | Find alternative component | • Move to component with better hygiene • Use fewer parts • Standardize on common components |
• Refactor could be expensive • May not return additional value • New grass may not be greener |
High | Low |
| Remove Component Entirely | Remove component; replace with your own code | • Remove external risk • Reduce project bloat |
• Slower innovation • Expensive • Potential vendor lock-in |
High | Low |
Component Upgrade
Upgrading the component to a safer, better version is the most effective remediation strategy. It is the least expensive option and can be automated (saving even more time and resources). The down side can include a possible longer wait time for the version to be released by the maintainer. If you have any transitive dependencies to worry about, those could be cumbersome to deal with. This option requires a slower, more methodical approach so as to not introduce more risk in the process of fixing the initial problem.
DIY Patch
One of the benefits with any do-it-yourself solution is that you are at the helm and control the code. With the DIY patch strategy, you are removing the problematic code from your execution path and replacing it with code that you write yourself. Hopefully you decide to share your coded solution with others who also have included the vulnerable component in their projects. Contributing to the community is, after all, what it’s all about. Another benefit is that this is a cheap and quick remedy, something that your leadership will especially appreciate.
Switch Components
While you most likely chose the vulnerable component initially because it provided precisely what your application required, there are often other components you could swap. Plus, if the swapped component has been hygiene, it’s a win-win. On the other hand, the grass isn’t always greener, and that’s one of the cons associated with this approach. The refactor process could be expensive in time and resources for your team and may not return any additional value.
Remove Component Entirely
If your organization can manage to lose the affected component, the benefit of removing it entirely from your application is that it removes the threat entirely. This is ultimately safest because, if you must retain the functionality the component provided, you can write and maintain the code yourself. While this can be a slower strategy to implement, you reduce the external risk.
Mitigation Strategies at a Glance
There are two main strategies to mitigate vulnerabilities, each with their own pros and cons based on an organization’s specific set of circumstances. They are as follows:
| Strategy | What it is | Pros | Cons | Cost/Effort | Risk |
|---|---|---|---|---|---|
| Waive | Keep component; mitigate as necessary | • Good option if not a direct risk | • Creates tech debt • Future upgrades more difficult |
Low | Medium-High |
| Baselining/Grandfathering | Accepting the legacy risk | • Reduce noise • Focus priorities |
• Ignores risk • False security • Creates tech debt |
Low | High |
Waiving
The most common mitigation strategy is to waive the risk. Some common reasons an organization would choose this approach are as follows:
- There is no current impact to the application (aka not exploitable).
- A grace period is needed to address the risk adequately.
- There is no current priority to remediate the issue (not in the budget).
- There is no path forward to resolve the issue.
When organizations choose to waive a violation, there are some best practices to follow:
- Use waivers instead of ignoring risks.
- Document a common format for waivers so they are understood and accountable.
- Avoid permanent waivers. Waivers should be reviewed periodically so that the reasoning remains consistent.
- Review waivers for high-threat risks more frequently than low-threat risks.
- When possible, consider switching components even if the application is not exploitable. Most often the component is still a risk by remaining in your application.
Baselining
For legacy or low-impact applications without active development, any remediation activity is probably outside the scope of the project and may not feasible. In these situations, the common strategy is to baseline and accept any existing risk and only remediate new risk. If you adopt this strategy, regularly review any baselined risk and develop a system to notify stakeholders when a new risk is identified.