Why DevSecOps?

Key Takeaways

• DevSecOps is an extension of DevOps that integrates security into the entire software development process.
• Software supply chain attacks have been increasing every year making it vital to proactively manage risk from your open source components.
• Including security through the development process moves an organization away from a, “scan and scold” approach to application security which can add friction between teams and slow down development.
• Addressing security concerns throughout the Software Development Lifecycle will save time, money, and help maintain customer trust.

Introduction

In our first technical article in the DevSecOps track, What is DevSecOps?, we explained that DevSecOps is an extension of DevOps—building quality in—that advocates for integrating application security measures into the entire software development lifecycle (SDLC), including open source software (OSS) component selection by software developers.

There are many compelling reasons for prioritizing OSS component management as part of your DevSecOps strategy, and we’ll discuss those in this article.

The (Increasingly Scary) State of the Software Supply Chain

(Open Source) Software Is Eating the World

In our current “software is eating the world” climate, developers are constantly “on the hook” to release software—with more and more complex features—faster than ever.

This mandate often involves reliance on OSS components. After all, why would you “reinvent the wheel” by writing your own proprietary code if there’s a freely available, widely-used component that provides the same functionality needed in your application?

In fact, according to Sonatype’s State of the Software Supply Chain report, open source software components make up more than 80% of most modern applications.

With open source becoming an increasingly prominent part of most software being developed, the role of software developers is also shifting from the more narrow definition of “coder” to a broader one that includes code assembly and code integration to achieve both the software platform and the business requirements within that platform.

And the software development process itself begins to look more and more these days like a traditional manufacturing supply chain—a software supply chain, if you will. In Sonatype’s IQ for Developers 100 course, we explain this analogy further:

Suppliers = OSS

Traditional manufacturing-based supply chains have trusted suppliers who build components used in the final assembly of the finished product. In the case of a software supply chain, open source projects supply some of the most common components to be used in the application being developed.

Warehouses = Component Repositories

Traditional supply chains have also streamlined logistics and sourcing by using large, centralized warehouses to store and distribute those components with more security and efficiency. Similarly, software developers now often rely on component repositories like the Central repository that Sonatype manages (Java), rubygems.org, pypi.org, the npm registry (javascript), and the NuGet Gallery (.net).

Manufacturers = Software Development Teams

To continue the metaphor, manufacturers are analogous to software development teams that are sourcing and consuming open source components used as building blocks in applications.

Assembled Product = Software Applications

Finally, the assembled and validated product in a traditional supply chain are comparable to the tested final builds of software applications that you make publicly available to your customers.

OSS Breaches Are Increasing (and Increasingly Sophisticated)

Here’s where it gets a little scary. There is a common perception that using open source components is safer, or that they’re less likely to be compromised, than proprietary software, possibly because of the “many eyes” approach of contributing to and reviewing OSS projects.

But according to the 2019 DevSecOps Community Survey, breaches tied to OSS components have increased 71% in the past 5 years.

Sonatype CTO Brian Fox details several of these attacks in his Forbes post Open Source Developers and Infrastructure Are the New Frontline of Security, with many more attacks occurring since its publication.
In addition to the number of attacks increasing, the attackers themselves are becoming more sophisticated in their approach, as Fox explains:

“When connected like pieces of evidence in an investigation, it becomes clear that we are in the middle of a systematic attack on the social trust and infrastructure used to distribute open source. In just a few years, we’ve gone from attacks on pre-existing vulnerabilities occurring months after a disclosure down to two days. Now, we are at the point where attackers actively create exploitable, directly monetizable (via crypto) conditions in these very components.”

One recent example of a highly sophisticated attack on an open source component is chronicled by Elisa Velarde in her blog post Corrupting the Software Supply Chain: Lessons from the Bootstrap-sass Attack. Here’s a brief summary of the turn of events that helped uncover the attack:

1. A developer who relies on the bootstrap-sass (RubyGems) component has a build fail.
2. The developer investigates and learns that the version used in his application was removed, and minutes later, a new version was added to the RubyGems repo.
3. Further investigation shows that the source code for the component library was not updated on GitHub, however, raising a red flag for the developer.

Thanks to the due diligence of this developer following a suspicious trail, the newly-added, vulnerable version of the bootstrap-sass component was removed from the RubyGems repo by the team the same day, as were the rights of the account believed to push the malicious version.

But this yank-and-replace, force-upgrade approach is becoming more common, with the ability to wreak havoc for anyone not adequately managing the component dependencies within their software.

How Are Your Open Source Software Components Being Managed (Or Not)?

According to the 2019 DevSecOps Community Survey, nearly 40% of software development organizations with 100 developers or less believe their current infosec teams/processes are slowing them down; nearly 53% of software development organizations with more than 5000 developers believe their current infosec teams/processes are slowing them down.

When integral security processes are at odds with a developers’ mandate to accelerate software delivery, a tension develops.

It’s not a controversial statement at this point in the evolution of the software development community that high-friction, legacy-infused processes have created a divide between security and development. What may be more controversial, however, is to follow the chasm that’s been created down into the trenches, looking at potential (if inadvertent) outcomes that may creep in when legacy processes begin slowing developers down.

For example, if a developer doesn’t have an easy, automated way of choosing a “good” OSS component over a “bad” component, they’re forced to make a subjective choice.

If that developer happens to make a “bad” choice (in a “scan and scold” type of environment), will there be any incentive for them to try again? Have they been equipped with the information necessary to make an informed choice on their OSS component selection?

Or what if they happen to make a “good” choice, instead? Phew, right? Why not stick to what worked, then? Is there any incentive to upgrade in that scenario, without specific information around vulnerabilities in other component versions to draw from?

Finally, what if the approval process itself takes so long, that the developer isn’t able to do anything about it when the determination from security finally comes back? The rework required to choose a new component that late in the game may be enough of a roadblock that they’re forced to stick with a “bad” component as well.

In this way, the lack of an integrated and automated open source governance strategy could inadvertently encourage developers to do the “wrong” thing (while also stifling innovation).

According to Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations, there is a better way:

“When building security into software is part of the daily work of developers, and when infosec teams provide tools, training, and support to make it easy to do the right thing, delivery performance gets better. Furthermore, this has a positive impact on security.”

The Business Case for DevSecOps

Nobody wants to be the next Equifax in the headlines

One of the most widely publicized open source breaches happened in 2017, when hackers gained access to millions of consumers’ personal info via the Equifax web site, which happened to rely on the popular Apache Struts open source web framework component.

It drew so many headlines because it affected so many people, and the reputation and trust of Equifax suffered greatly.

Gaining (and keeping) customers’ trust is not an easy task for any company, and minimizing the risk of losing their trust is at the top of many organizations’ priority lists in this climate of security breaches and privacy-related issues.

The average time to exploit continues to decrease dramatically, as seen in the graph below, so organizations have a much narrower window to address these security breaches.

Unfortunately, the impetus for some organizations to prioritize security may only be when a security breach of their own occurs. But wouldn’t a better strategy be prevention, so that customers’ trust—in addition to security—is never breached in the first place?

Security as a Competitive Differentiator

In her All Day DevOps talk Why Does Security Matter for DevOps?, Caroline Wong discussed how security is not only becoming table stakes for enterprise-level, compliance-minded organizations, but also a competitive differentiator:

“For DevOps companies, securing software can actually become a competitive differentiator. So, security is a feature that’s marketed, and it promotes necessary trust with customers and partners. So if your peers and your competition are focused on application security and you’re not, you might actually risk falling behind.”

Development Cost Savings

It may seem counterintuitive at first, and even possibly at odds with developers’ ability to maintain a certain velocity, but integrating automated open source governance earlier in the software development process can translate to cost savings overall:

• Developers will spend less time researching and vetting OSS components if they have automated tooling that does much of this legwork for them.
• When security-related code reviews happen at the end of the development cycle, any findings that involve re-work or technical debt incurred at that stage will be more costly in the long run than if the security issues were found earlier in the cycle.
• Further, if a security vulnerability is found in your application after it’s already in production, the costs associated with the “all hands on deck” approach to remediation can be astronomical.

According to Accelerate authors, remediation time is significantly reduced with the adoption of a DevSecOps workflow:

“We found that high performers were spending 50% less time remediating security issues than low performers. In other words, by building security into their daily work, as opposed to retrofitting security concerns at the end, they spent significantly less time addressing security issues.”

With bad actors increasingly “poisoning the well” as well as retaining customer trust, remaining competitive, and keeping costs from running over on the line, the business case for DevSecOps begins to shift further…into focus.

Up Next: In the next article, we’ll discuss a staged DevSecOps “maturity model” that focuses on OSS component management.

Sources

2019 DevSecOps Community Survey

State of the Software Supply Chain Report

Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations by Nicole Forsgren PhD, Jez Humble, and Gene Kim

Corrupting the Software Supply Chain: Lessons from the Bootstap-sass Attack by Elisa Velarde

Open Source Developers and Infrastructure Are the New Frontline of Security by Brian Fox

Why Does Security Matter for DevOps? by Caroline Wong on All Day DevOps talks

Additional Resources

Visit Intro to DevSecOps for an interactive course.

Talk to Us

Have more questions or comments? Learn more at help.sonatype.com, join us in the Sonatype Community, and view our course catalog at learn.sonatype.com.

And visit my.sonatype.com for all things Sonatype.