Supply chain risk: Addressing a multitude of single points of failure

It's well understood that the technology supply chain introduces risk, but until recently, the focus has been on people and processes, leaving the technology itself as a major visibility gap. To effectively manage supply chain risk, government organizations must understand and address the full scope of the supply chain. As recent attacks have demonstrated, that extends to the software and update process as well.

When considering supply chain risk, an attack during product transport can cause irreparable harm. However, physically tampering with hardware is not scalable. Manipulating the software inside hardware (firmware), on the other hand, very much is. In the Sunburst campaign, attackers delivered a malicious backdoor to over 18,000 SolarWinds customers by compromising the authorized software update infrastructure. This is similar to the previous ShadowHammer attack, where compromised ASUS update servers were used to push malware to hundreds of thousands of customers. In both cases, the updates were properly signed and appeared valid.

Scope explosion makes it worse

Every single laptop, desktop, server or other information technology device is composed of dozens of components. Each of those components has a supply chain of its own. At any point in the process, an attacker could modify or insert additional code, or the code could have been created with an accidental vulnerability, such as a logic bug or buffer overflow, or even a deliberate backdoor. As is often the case with network devices, hard-coded username and password backdoors end up being the initial vector that allows adversaries to gain access to additional attack surfaces on the device operating system or in the organization.

Risk naturally increases as the attack surface grows, but supply chain risk is also heightened by its concentration. Every piece of modern technology is built upon assumptions and abstractions from lower layers. One issue anywhere in the supply chain of any component can usually break the assumptions of every operating system and application. Given the scope of the supply chain, there isn't just one single point of failure – there are hundreds, possibly thousands, of single points of failure in an IT environment.

To make matters worse, one failure can equate to multiple compromises. While the components themselves are discrete, the pieces of software within each component may be used in components across multiple products from different manufacturers. As a result, if a software library is compromised anywhere, it's compromised everywhere. The same is true of software libraries included in firmware, many of which are common to many components and rarely updated.

Similarly, supply chain risks are particularly impactful to large, homogeneous environments, such as data centers, cloud infrastructure and virtual environments within the enterprise. A single vulnerability in a Baseboard Management Controller (BMC) can lead to the compromise of the entire data center.

The implications of supply chain risk

Supply chain security is not a "point in time" affair. Manufacturers can build good security mechanisms, but systems evolve, vulnerabilities surface and attackers improve their methods. Bottom line: something trusted today may not be trustworthy tomorrow.

Given the vast quantity of components that make up any given system, there are countless opportunities for vulnerabilities to show up. There can be a high concentration of critical risk duplicated a thousand times through the long supply chain, and any one of them is enough to cripple infrastructure.

While the federal government has a number of processes for validating the supply chain, these are not necessarily scalable or practical. One cannot conclude that a product made in the United States is robust against these issues; the problem is much more complicated than manufacturing origin. Even if a system is deconstructed, analyzed and reconstituted, the need for software/firmware updates usually invalidates that work. Systems designed to address this complexity with continuous and rigorous verification are badly needed but far off. There are "turtles all the way down," and they bite.

How to address supply chain risk

When you find yourself stuck in a deep hole, the first step is to stop digging. This means establishing a baseline of visibility into the firmware and hardware present in every component of every device. Without this visibility, the risk to the enterprise (or the mission) cannot even be quantified and registered, let alone managed down to acceptable levels. Once the organization has visibility, it can begin to apply the same processes used to manage risk at the OS, network and even physical or business layers. Deploying updates, patch and configuration management can and should be made to work for each component inside of each device.

Managing supply chain security with a vendor-independent tool allows organizations to centralize risk management efforts for all the systems in their heterogeneous environment. Independent checks also provide added assurance and eliminate the risk of vendor lock-in. Manufacturers have a key role in this story, too. By building transparency, secure attestation solutions and sharing BOM integrity data with 3rd party solutions, they can enable this increased visibility that is so badly needed. Building upon such increased visibility, both public and private organizations can work together to create a reasonable foothold in understanding and responding to supply chain risk.

Supply chain risk isn't new. But the supply chain is becoming a bigger target as attackers look for easier ways to increase the severity and scale of their attacks. Government organizations need policies and procedures that think holistically about the supply chain and implement a pragmatic approach to reducing supply chain risk, one that extends to and includes the continuous monitoring of supply chain threats in the operational environment.