CSO Perspectives is a weekly column and podcast where Rick Howard discusses the ideas, strategies and technologies that senior cybersecurity executives wrestle with on a daily basis.
The year we realized digital supply chains were a thing.
Because of the various intrusion kill chain models (Lockheed Martin’s, MITRE’s, Diamond’s), network defenders have known for at least a decade that cyber adversaries have to string a series of actions together in order to accomplish their goals. One of those actions is to establish a beachhead somewhere on the victim’s network. Gaining that foothold will allow them to start moving laterally to find the data they have come to steal, destroy, or hold hostage.
Traditionally, in order to counter that important first step, network defenders have focused on direct attacks on the most important and material information and the digital islands where we store it all (traditional perimeter, endpoints, SaaS, IoT, and IaaS/PaaS). The main attack techniques adversaries have used to gain that initial access have been phishing and watering hole attacks, and preventing those have been hard enough. Until recently however, most of us have totally ignored an indirect way to establish a beachhead, a back door way, going around our front line defenses through the supply chain.
To be fair, most of us knew that this indirect attack vector was a potential problem. We just hadn’t seen it pop up in traditional attacks that often. But we absolutely knew that attacks following these back alley supply chain paths could be material to the business. After all, supply chain management in the physical world has been a material part of the business since ancient times.
A little supply chain history.
In the industrial age, Fredrick Taylor, the founder of industrial engineering, wrote “The Principles of Scientific Management” in 1911 that sought to apply scientific rigor to running business operations. This is the same year that UPS opened its doors and global supply chains started to take shape.
By the 1940s and during World War II, the military on all sides became interested in how to efficiently move large pallets of materials around the world and started calling it “Supply Chain Engineering.” When computers entered the picture in the 1960s, researchers began automating the processes of warehousing, material handling, and freight transportation. By the 1970s, DHL and FedEx had started their operations, and JC Penney built their first just-in-time warehouse management system (WMS).
In 1982, a Booz Allen Hamilton consultant, Keith Oliver, coined the term “supply chain management” defining it as “the process of planning, implementing, and controlling the operations of the supply chain with the purpose to satisfy customer requirements as efficiently as possible. It spans all movement and storage of raw materials, work-in-process inventory, and finished goods from point-of-origin to point-of-consumption.” If this sounds familiar, this is also how Gene Kim described DevOps in his Cybersecurity Canon Hall of Fame book, “The Phoenix Project.”
Preparing for Y2K in the early 1990s, researchers invented Enterprise Resource Planning (ERP). According to Clay Halton at Investopedia, “Y2K was commonly used to refer to a widespread computer programming shortcut that was expected to cause extensive havoc as the year changed from 1999 to 2000.” The IT industry spent tons of resources to make sure a digital armageddon didn’t happen. When it didn’t, the eternal chicken-and-the-egg question emerged: did our Y2K efforts prevent the disaster or did we all over react and we really didn’t have a problem after all (A question for the ages)?
But in 1998, Microsoft rolled out Windows Update for it's Windows 98 operating system for the first time. Some technology vendors had rolled out this over-the-internet update idea in various forms before this, but I can make a strong case that this is the point when we all needed to start worrying about digital supply chains. At this point, everybody had some version of the Windows Operating System running somewhere. This was the first time that a potential digital supply chain attack could go global.
Before Windows Update, we all did software updates via floppy disks and later CDROMs. Bad guys could still insert trojan horses into vendor code, but the new code distribution system was via snail mail. The potential impact was present but it wasn’t instantaneous. After Windows Update came out, the practice of upgrading your software via the internet started to become acceptable and made upgrades almost instantaneous. Today it's common practice. Most people don’t think twice about clicking the upgrade button on whatever software they are running.
By the 2000s, the concept of managing supply chains became so important that in 2005, the “Council of Logistics Management” changed its name to the “Council of Supply Chain Management Professionals.” In the digital space, GitHub launched its first platform in 2008 and that same year, Synopsys released its first version of the Building Security In Maturity Model (BSIMM).
GitHub is a cloud-based service that helps developers manage their code storage via Git, an open-source version control system created by Linus Torvalds in 2005. Hackers have been leveraging Git to exploit the Log4J vulnerability. BSIMM quantifies software security development best practices by interviewing commercial vendors who have mature development houses. One of BSIMM’s best practices is managing Software Bill of Material (SBOMs) for all of the software packages running in your organization.
I mention SBOMs because that concept has been bouncing around the industry for years as one way to reduce the risk of digital supply chains. But, it hasn't really gained that much traction until last year. U.S. President Biden mandated that all Federal Civilian Executive Branch Agencies (FCEB) and Key Players start using SBOMs by the spring of 2022 in his Executive Order 14028.
What is an SBOM?
An SBOM is a formal record containing the details and supply chain relationships of various components used in building software. According to the NIST Cybersecurity Framework, “If an organization does not know what its software contains, it should assume that the software is compromised and develop an appropriate risk management plan.”
Today, very little software is completely original. On average, 75 percent of a software product is open-source code, meaning developers are using existing, commercially available software components to create new products. This presents a cyber-risk management problem because customers typically receive software products without understanding the nested software contained within them.
In September 2020, the U.S. Department of Defense (DoD) published their initial vision for the Cybersecurity Maturity Model Certification (CMMC) program that mandates several SBOM milestones. The very next year, the Software Package Data Exchange® (SPDX®) specification became the international open standard for security, license compliance, and other software supply chain artifacts. In other words, they became the official SBOM standards body. Despite only being internationally recognized for a short while, companies like Intel, Microsoft, Sony, and VMware are already using the SPDX standards to communicate SBOM information.
SPDX was not an overnight invention though. It was the result of ten years of collaboration from vendors across the Software Composition Analysis (SCA) space; tools that assess open source software, code libraries, and containers, to provide a unified view of risks and remediations and offer strategies to keep this kind of software up to date.
With the U.S. government mandating SBOM requirements, vendors that sell to the U.S. government will have to comply. It’s tough to predict these things but once government contractors routinely provide SBOM information, that capability becomes a discriminator against other software vendors. In the commercial space, why would you pick a vendor who doesn’t provide SBOM telemetry when other vendors are available who do? If this works out, the Presidential Directive could fast track SBOMs as an existing standard to protect against supply chain vulnerabilities.
Zero Trust as “the” strategy to protect the digital supply chain.
In the physical world, the supply chain is somewhat linear: Order parts, ship to a consolidation port, sail to a receiving port, transport goods to a final destination. Technologies like machine learning, the Internet of Things, automation, and sensors are transforming the way companies manufacture, maintain, and distribute new products and services. Businesses are calling it Industry 4.0 and it’s totally built on the modernized supply chain. But it's still mostly linear.
In the digital world, the supply chain is not linear at all. According to one analyst at Oracle, “It’s a complex collection of disparate networks that can be accessed 24 hours a day. At the center of these networks are consumers expecting their orders to be fulfilled―when they want them, the way they want them.” In many cases, vendors deliver software and software updates right to the home or to the business.
When you look at first-principle strategies that will have the greatest impact on reducing the probability of material impact due to a digital supply chain attack, we all have to turn to zero trust. From the original John Kindervag white paper, we implement the zero-trust philosophy by reducing the attack surface as much as possible and allowing employees and contractors access to the data and workloads they need to do their jobs and nothing else. For the digital supply chain, two tactics emerge that we have to get right.
The first zero-trust tactic is that we must limit access to any running application, whether vendor-supplied or anything we developed ourselves. For example, in the Solarwinds attack of last year, the main Solarwinds product is a network management platform called “Orion.” Hackers broke into the Solarwinds’ network and inserted a remote access trojan (RAT) into the Orion software package. When Solarwinds’ customers downloaded the next update, they downloaded the RAT. After the install, victims were running the backdoor code on their Orion platform. Hackers leveraged the RAT to gain access to the multiple Solarwinds customers running that application. But damage didn’t come from that initial beachhead. Once they were on the Orion platform, the hackers moved laterally, leveraging credentials and access granted to the Solarwinds application.
In a zero-trust environment, Orion credentials shouldn't have access to anything important most of the time. In the cases where it does need administrative privileges, those escalations shouldn’t be allowed without specific approval controls in place. That’s reducing the attack surface. That’s limiting access. That’s zero trust.
The second zero-trust tactic is a robust deployment of SBOMs for every vendor product you deploy and for every open source software component that you use. SBOMs by themselves won’t reduce the probability of material impact from your vendors and from these open source libraries, but they are an essential building block to creating a zero-trust environment for the software that we all deploy. In the same way that identity management helps network defenders have complete visibility of every person and device that is running on your network, SBOMs provide that intelligence for the software that you are using. When issues arise like the Log4J vulnerability, you’ll know immediately if you’re impacted because of your SBOM. Again, it doesn’t solve the problem, but at least you have a starting point.
The future of digital supply chains.
Digital supply chains are not a new vector. They are an old vector that has been around since at least the late 1990s, and probably well before. Nation-state hacker groups have used them for decades. But, the technique got noticed in 2021 because of attacks against a couple of IT vendors, Solarwinds and Accellion, and because of the Log4J vulnerability stored in Github code repositories.
To defend against attacks that arrived through these indirect vectors, the zero-trust strategy is likely your best bet in reducing the probability of material impact. SBOMs can help, but they are at least five-to-ten years away from universal acceptance by all software vendors and Github deployments. President Biden’s Executive Order might speed that timeline up a bit, but I'm not holding my breath. There’s a lot of work that needs to be done by everybody.
In the meantime, the best way to reduce the threat of digital supply chains is to get an inventory of all applications running on your network and the people and machines that have to connect to them for business operations. Once that is in place, you can start implementing some role-based rules about who has access to what.
“A Software Bill of Materials Is Critical for Comprehensive Risk Management,” By Dr. Georgianna Shea, Foundation for Defense of Democracies, 2021.
“Association for Supply Chain Management (ASCM)” Ascm.org, 2021.
“Best Practices for a Secure Software Supply Chain.” by JonDouglas, Microsoft, 4 November 4, 2021.
“Controlling Privacy and the Use of Data Assets.” Security Boulevard, 4 August 2021.
“Framing Software Component Transparency: Establishing a Common Software Bill of Material (SBOM): NTIA Multistakeholder Process on Software Component Transparency Framing Working Group,” by Michelle Jump, Nova Leah, and Art Manion, Framing Working group, 12 November 2019.
“GitHub - Notaryproject/Notaryproject: Requirements and Scenarios of the Notary v2 Project” notaryproject, GitHub, 16 December 2021.
“History of Supply Chain Management,”Business-Essay.com, 2021.
“History of Supply Chain Management.” Flash Global - Supply Chain Logistic, August 7, 2015.
“Inside the Journey of a Shipping Container (and Why the Supply Chain Is so Backed Up).” Wired, 2021.
"No More Chewy Centers: Introducing The Zero Trust Model Of Information Security," by John Kindervag, Forrester, 14 September 20210.
“ORAS Artifacts Specification.” oras-project, GitHub, 3 December 2021.
“Ratify Framework Overview,” GitHub, 2021.
“SBOM FAQ,” by NTIA Multistakeholder Process on Software Component Transparency, NYIA, 16 November 2020.
“SCOR Digital Standard | ASCM.” Ascm.org, 2021.
“Securing the Kubernetes Software Supply Chain.” by Simon Bisson, Simon, InfoWorld, December 15, 2021.
“Software Bill of Materials (SBOM) (Noun),” by Rick Howard, Word Notes, The CyberWire, November 2, 2021.
“SolarWinds Attack Explained: And Why It Was so Hard to Detect.” Lucian Constantin, CSO Online, 15 December 2020.
“SPDX Becomes Internationally Recognized Standard for Software Bill of Materials,” by The Linux Foundation, Prnewswire.com, 9 September 2021.
“Supply Chain Management: Strategy Planning and Operation,” by Chopra, Sunil, and Peter, Mendi, 3 ed. Upper Saddle River, New Jersey: Pearson Education, 2007.
“Taking Windows 98 for a Test-Drive.” By John Gartner, TechWeb, WayBackMachine, 25 June 1998.
“The History of Supply Chain Management,” by Supply Chain Digital, 2021.
“The Minimum Elements For a Software Bill of Materials (SBOM) Pursuant to Executive Order 14028 on Improving the Nation’s Cybersecurity,” by The United States Department of Commerce, 12 July 2021.
“The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win,” by Gene Kim, Kevin Behr, George Spafford, Published by IT Revolution Press, 10 January 2013.
“The Principles of Scientific Management,” by Frederick Winslow Taylor, Published by Cosimo Classics, 1911.
“The SCOR Model for Supply Chain Strategic Decisions | SCM | Supply Chain Resource Cooperative (SCRC),” Supply Chain Resource Cooperative, North Carolina State University, 27 October 2004.
“The Untold Story of Github,” by Saurabh Mhatre, Medium, 24 October 2016.
“What Is GitHub? A Beginner’s Introduction to GitHub.” Kinsta®, May 28, 2021.
“What Is Supply Chain Management? (SCM).” Oracle.com, 2021.
“What Is the BSIMM and How Does It Work?,” Synopsys.com, 2020.
“What Is Y2K?” Investopedia, 2022.
“What to Know about Software Bill of Materials.” Samantha Schwartz, Cybersecurity Dive, 20 September 2021.
“Why Christmas Gifts Are Arriving on Time This Year.” The New York Times, 2021.
“Why It’s Time for Cybersecurity to Go Mainstream,” by the CyberWire Staff, CyberWire-X, 26 September 2021.
Supply Chain Timeline.
- Fredrick Taylor, the founder of industrial engineering wrote “The Principles of Scientific Management” in 1911, targeted the process improvement of manual loading in his work.
- Global supply chains started to take shape
- UPS opened their doors.
1940s and 1950s
- Research on Pallet and pallet lift mechanization began to obtain better warehousing space, racking and layout.
- The “unit load” concept became popular by utilizing intermodal containers together with ships, trains, and trucks to transport them. This set the stage for supply chain globalization.
- The military began Operations Research of analytics during WWII for logistical solutions
- The industry invents “Supply Chain Engineering.”
- IBM developed the first computerised inventory management and forecasting system.
- In 1963, the National Council of Physical Distribution Management organization became the field leader particularly due to the advent of computing in the 1960s and 1970s and the resulting paradigm shift.
1960s and 1970s
- Time-sensitive freight transportation steered toward trucking rather than railroad
- “Physical Distribution” became a joint need (warehousing, material handling, and freight transportation).
- In the late 1970s and early 1980s, Georgia Tech created the Production and Distribution Research Center, Material Handling Research Center, and the Computational Optimization Center. Each center focused on different aspects of what could be done with computing technologies
- DHL joined the growing number of logistics providers.
- FedEx joined the growing number of logistics providers.
- JC Penney created the first real-time warehouse management system (WMS). This was a game changer. Updating stock inventory in real-time, Jit reduced time spent looking for stock and allowed the company to focus on growing the business.
- Personal computing began a logistics transformation with marked improvements in supply chain management.
- Innovations included the Production and Distribution Research Center using joining map interfaces with optimization models for supply chain design and distribution planning.
- The Computational Optimization Center built Massive algorithms for airline schedules
- Systems are costly to acquire and complicated to operate, yet absolutely crucial to corporate profits.
- Booz Allen consultant Keith Oliver coined the term ‘supply chain management, using the term in an interview with Arnold Kransdorff of the Financial Times, on 4 June 1982. Oliver is a British logistician. Oliver defined it thus: “Supply chain management is the process of planning, implementing, and controlling the operations of the supply chain with the purpose to satisfy customer requirements as efficiently as possible. It spans all movement and storage of raw materials, work-in-process inventory, and finished goods from point-of-origin to point-of-consumption.”
- The National Council of Physical Distribution Management became the Council of Logistics Management (CLM) “to reflect the evolving discipline that included the integration of inbound, outbound and reverse flows of products, services, and related information.” Beforehand, the term had been mostly reserved for military logistics.
- Enterprise Resource Planning (ERP) began appearing to integrate the various, disconnected company databases as a failsafe for Y2K preparedness.
- ERP software identified planning and integration needs for logistics components resulting in a new generation of “Advanced Planning and Scheduling (APS)” software.
- Globalized manufacturing such as the growth of manufacturing in China in the mid 1990s popularized the term “supply chain”.
- First cobot is invented, A cobot, or collaborative robot, is a robot intended for human interaction. They were invented in 1996 by J Edward Colgate and Michael Peshkin, professors at Northwestern University. Their invention sprang from a 1994 General Motors initiative to find a way to make robots or robot-like equipment safe enough to team with people.
- Microsoft added the Windows Update feature to the Win 98 operating system; a diagnostic website that checked PCs for the latest Windows components and hardware drivers.
- Linus Torvalds introduces Git, an open-source version control system.
- GitHub launched its first platform.
- Synopsys released its first version of the Building Security In Maturity Model (BSIMM).
- The National Telecommunications and Information Administration (NTIA) convened an open, transparent multi-stakeholder process on software component transparency.
- The U.S. Department of Defense (DoD) published their initial vision for the Cybersecurity Maturity Model Certification (CMMC) program that mandates several SBOM milestones
- President Joe Biden’s executive order (E.O.) on cybersecurity, E.O. 14028, mandates that all Federal Civilian Executive Branch Agencies (FCEB) and Key Players meet or exceed specific cybersecurity requirements. Specifically, they will deploy a minimum SBOM program by the spring of 2022.
- NTIA issued its guidelines for the minimum elements for an SBOM.
- The Software Package Data Exchange® (SPDX®) specification became the international open standard for security, license compliance, and other software supply chain artifacts. In other words, they became the official SBOM standards body.