Cybersecurity moneyball: First principles applied to the workforce gap.
The infosec community has been talking about the cybersecurity workforce gap for well over a decade. What I mean by “workforce gap” is the number of unfilled cybersecurity jobs that exist at any particular time. The earliest mention I could find of network defenders’ awareness of the problem came from a report by the Center for Strategic and International Studies (CSIS) in 2010, “A Human Capital Crisis in Cybersecurity: A Report of the CSIS Commission on Cybersecurity for the 44th Presidency.”1 In that report the authors claimed that the shortfall was between 9,000 and 29,000 depending on how you counted the jobs, and that this total was just for the United States. In 2022, the International Information System Security Certification Consortium (ISC)2 said that the global cybersecurity workforce gap was 3.4 million people. That’s over 116 times the number calculated by CSIS. Clearly, we have a problem finding qualified people. And it’s not like we haven’t tried to fix the problem. I mean, it isn’t like we weren't aware. There has been a steady drumbeat in public forums since 2010 of the situation getting worse each year.
Academic and certification programs have responded. It feels like most colleges today compared to 2010 offer some kind of learning path to cybersecurity and have been churning out graduates for a while now. And there are more potential certification classes available today than there has ever been. If that’s so, then why is the workforce gap continuing to grow? And why have we all heard the horror stories where a newly minted cybersecurity graduate can’t get a job?
The problem as I see it is that we continue to hire cybersecurity talent and train our existing teams in the same way we started doing it back in the day (say early 2000s). As with that old chestnut, insanely, we expect to close that gap with this approach even though the evidence is telling us that the problem is getting worse.
Our hiring and training method is simple. We focus on the individual. When we hire, we are looking for the all-star; somebody with 25 years of experience, a technician with 17 certifications, and an employee willing to work for a buck fifty an hour. No wonder we can’t find anybody. When the organization trains its own people, leadership is all for it. But, we send the individual. We pay upwards of $3000 for an employee to attend a class or a conference to get up to speed on some new thing. Most times, we ask the individual what he or she wants to learn, not as a training task but as a perk for being part of the organization. We don’t really have a team training strategy at all. We focus on the individual.
With these tactics, we struggle to bring on talent with the skills we actually need and we are surprised when the training impacts one employee, not the overall organization. In other words, after the conference, we have one employee who understands the basics of Chaos Engineering, let's say (a first principle tactic supporting our resilience strategy), but the infosec team is still mostly in the dark. One side effect is that the all-star coming into the organization and the all-star we create are prime candidates to be pilfered by some other organization who is willing to pay more money.
Sending individuals to training then seems like a losing strategy, and yet we continue to do it. If we want to implement one or more of our first principle strategies (zero trust, intrusion kill chain prevention, resilience, risk forecasting, and automation),2 perhaps we need to shift our focus away from the individual and towards training the team.
The Oakland A’s.
I'm reminded of one of my favorite movies, Moneyball, starring Brad Pitt and Jonah Hill, released in 2011 and based on the 2003 book of the same name by Michael Lewis. Lewis tells the story of how the Oakland A’s, an American Major League Baseball (MLB) team, adopted a radical new approach to fielding players. In 2002, the A's had a payroll of approximately $42 million, while the New York Yankees, their arch nemesis, had a payroll of around $126 million.3 That meant that the Yankees could buy the best players in the game and the A’s could hardly compete. As a response, the A’s General Manager, Billy Beane (played by Pitt in the movie), adopted the sabermetrics model invented by Bill James.4
Before sabermetrics, professional baseball teams chose players solely through observation. They used scouts, people who had been involved in the game for years, to subjectively evaluate potential players based on the scout’s experience. They looked for intangibles like bat speed, power, home run potential, attitude, personality, and whether or not the player had a good-looking girlfriend. In the movie, Jonah Hill (playing Peter Brand, a player analyst working for Pitt with a background in economics) says, “There is an epidemic failure within the game to understand what is really happening. And this leads people who run Major League Baseball teams to misjudge their players and mismanage their teams.”5 And I believe that a similar situation has been happening in cybersecurity since the beginning. We’re enamored with the superstar (17 certs, 25 years experience) and not with the aggregate skill set of the team. Consequently, only the rich teams (the Fortune 500s and the Silicon Valley companies) can afford to pay them.
The sabermetrics model uses data and statistics to find the exact skills that a team might need. And Billy Bean reduced his problem of how a low-payroll MLB team can compete with the high-payroll teams like the New York Yankees down to one atomic first principle: the most valued skill is not homerun percentage or whether or not the player has a good looking girlfriend, but players getting on base.6 He decided to build a team on that first principle.
In the movie, Hill tells Pitt, “Your goal shouldn’t be to buy players. Your goal should be to buy wins. In order to buy wins, you need to buy runs.”5 And in order to buy runs, you want players who routinely get on base. In the cybersecurity world, you don’t want to buy the superstar. You want to buy and train an aggregate team proficient in our first principles; not one person who knows everything, but a team that can collectively do it all.
Prior to the 2002 season, MLB teams with large payrolls stole the A’s top-three players in terms of perceived talent and actual salary: Jason Giambi (New York Yankees), Johnny Damon (Boston Red Sox) and Jason Isringhausen (the St. Louis Cardinals).7 Most pundits in the sports world wrote off the A’s season, believing they couldn’t recover from the losses. But Beane had a different idea. In the movie, Hill tells Pitt, “I think it's a good thing that you got Damon off of your payroll. I think it opens up all kinds of interesting possibilities. The Boston Red Sox see Johnny Damon, an All Star who's worth seven and a half million dollars a year. When I see Johnny Damon, what I see is an imperfect understanding of where runs come from. The guy's got a great glove. He's a decent lead off hitter. He can steal bases, but is he worth the seven and a half million dollars a year that the Boston Red Sox are paying him? No. No. Baseball thinking is medieval. They are asking all the wrong questions.” 8
In one movie scene, Pitt is sitting around the table with his collection of old guy scouts. They are still trying to pick players based on their intuition. Pitt pipes up in frustration. He says that the scouts are still trying to replace Giambi and the others with similar players and there is no way to do it with their payroll. But what they might be able to do is replace them in the aggregate. The three departing player’s average on-base percentage was .364. What they should be looking for are three relatively cheap players whose on-base percentage is the same.
In the cybersecurity world, the relatively cheap player is the newbie cybersecurity employee just coming out of college, or the government worker transitioning to the civilian world. It's also the relatively low-level employee already on the staff. All have little experience compared to an all star, but most have an aptitude and a desire to learn. Instead of hiring the superstar with the 17 certs for a lot of money, or training one of your existing superstars to be even more super, we could instead make the entire infosec team better by hiring and training the needed skill sets in the aggregate, just like Billy Beane did with the A’s.
Did Sabermetrics work for the A’s?
According to Garrett Chandler at the Modern War Institute at West Point, the A’s finished their 2002 season with a hundred and three wins, one more than they did the previous year with their three superstars: Giambi, Damon, and Isringhausen. And although it’s true that they haven’t won a World Series since they started the program, “they have been in the playoffs eleven times in the past twenty-two years (2000–2021), tied for fifth most in the league and have constantly put themselves in a position to win.” 9 And I would say that they did this against teams with a much bigger payroll, and a league of teams that started using the same sabermetric methodology after 2002 because of the A’s success with it. That is extraordinary. Further, another low payroll team (lower than the A’s payroll) that uses a similar system, the Tampa Bay Rays, have made it to the World Series twice in just over a decade. They lost both times, but they made it to the show.10 There’s no question that the Sabermetric analytical system has made lower payroll teams more competitive in the league. I believe it’s time for the network defender world to take it out for a spin.
“Don’t buy superstar players. Buy down risk instead.”
The way that infosec leaders train existing employees today, they focus on the individual’s needs. When they acquire talent today, they ask potential employees if they have 25 years of experience and 17 certs. As Jonah Hill said in the movie, “We are asking all the wrong questions.”8 If that’s true then, what are the right ones? What is the cybersecurity equivalent of the A’s buying runs and not people.
I was talking to my friend, Joe O'Brien, about this recently. He is the co-founder of Orion Cyber, where he helps organizations identify, quantify, and prioritize cyber risk. He said that, from his perspective, security leaders should seek to buy down risk, not buy superstars. When I heard that, the entire idea locked into place for me.
As you have heard me say in these essays, in the CSO Perspectives podcast, and now in the Cybersecurity First Principles book, the ultimate cybersecurity first principle, the thing that all of us are trying to do, is to reduce the probability of material impact to our organization due to a cyber attack. When it comes to training and hiring, the network defender’s goal shouldn’t be to buy or build superstar players. In order to buy down risk, you need to enhance the team’s ability to pursue the ultimate first principle. It’s a subtle distinction but an important one. The team skills you need to accomplish that are different depending on the follow-on strategies you adopt (zero trust, intrusion kill chain prevention, resilience, automation, and risk forecasting) but the ultimate goal should be to reduce risk.
Think of your infosec team as equivalent to the Oakland A’s in terms of talent acquisition and training. The thing that the Oakland A’s and all the MLB teams have going for them is a deep treasure trove of player statistics going all the way back to the beginning of the league in 1876. When you have that kind of data store, there are all kinds of ways to slice and dice the information that might provide useful insights to the ultimate first principle. For the cybersecurity community though, we don’t have that.
According to Statistica, there were approximately 4.6 million infosec professionals in the world in 2022.11 Unfortunately, we don’t have a database that shows what skills each of those “players” has. The network defender world is so new (last 30 years) and the technology we use to do our jobs changes so fast, that it’s tough to get a handle on everything that everybody is doing.
The closest we have come, I believe, is the Workforce Framework for Cybersecurity (the NICE Framework) developed by the U.S. National Institute of Standards and Technology (NIST). NICE stands for the National Initiative for Cybersecurity Education and the framework is “a reference taxonomy—that is, a common language—of the cybersecurity work and of the individuals who carry out that work” in cybersecurity.12 The framework groups the kinds of cybersecurity jobs we all have in big overarching categories:
- Oversight and Governance (OG)
- Design and Development (DD)
- Implementation and Operation (IO)
- Protection and Defense (PD)
- Intelligence (IN)
- Cyberspace Effects (CE)
It provides typical job titles (work roles), job descriptions, and the knowledge that a network defender must have in order to do each job. NIST publishes a comprehensive spreadsheet of all that information on their website.13 That work product by itself is invaluable as a reference tool for security leadership. When you’re writing job descriptions or employee performance reviews, why create everything from scratch when you have a ready-made consensus collection of the job descriptions and associated tasks already available? At least you can use it as a first draft and modify it later.
That said, if we are indeed trying to buy down cyber risk by improving the team’s skillset, the first task would be to map the NICE categories to our first principles. We would want to identify all the job categories and tasks associated with the first principle strategies and tactics that we are pursuing. I haven’t done that yet for all the NICE categories and for all the first principle strategies. That’s a future project for me for the summer of 2023. But, if you are playing along at home, you could use the “Road Map” on the First Principles’ book website as a handy cross check visual.14
For example, from the Road Map, I can see that for our zero-trust strategy and the tactic of vulnerability management, the NICE Framework lists the Vulnerability Assessment Analyst (PR-VAM-001). That employee performs system and network assessments and identifies where they deviate from acceptable configurations. From the NICE spreadsheet, there are 36 knowledge areas that apply, 12 specific skills, and four described abilities associated with that job.
My future task then is to identify all of those items for each tactic described on the First Principles Road Map. That’s the first step. The second step is to evaluate the team against the knowledge areas, skills and abilities. Assess how good the team is at everything. Once you have that data, you can then prioritize the team’s training agenda that will buy down the most risk.
That all sounds like a lot of work, and it is. But it’s work that needs to be done if you buy into the whole cybersecurity first principle idea as applied to workforce development. This is the entire reason using first principles is important. Up to this point in our collective cybersecurity history, team training hasn’t even popped up as something that we all need to do. Instead, we have focused on the individual as a superstar for hiring purposes insisting that we only consider the most highly qualified people available. For existing team members, security leadership has, for the most part, abdicated any kind of team strategy in favor of improving individual superstars. When you consider the problem of 3.4 million, and growing, open positions, in the cybersecurity workplace today, clearly those strategies aren’t working.
What I'm advocating is learning from the example of Billy Beane’s Oakland A’s: fielding a team designed to win games. He realized that the first principle for building competitive professional baseball teams was not to buy all star players but to build an all star team in the aggregate using relatively cheaper and overlooked players, and concentrating on using on-base percentage as the stat to rotate on. I'm suggesting that security professionals can do the same thing by rotating on first principle strategies and tactics. The implication, though, is that we have to adjust our mindset away from hiring and training superstars and be willing to field a team in the aggregate. That means tapping into the pipeline of new graduates coming out of college with no experience. It means taking a chance on a young potential employee with no certifications but lots of aptitude. It means developing a well thought-out and consistent training plan for your team, a workforce development strategy, that will allow you to buy down risk. And it means creating the team training tactics that will support that strategy. After all, you can’t really implement a first principle intrusion kill chain prevention strategy without a team that knows what that is, and how it can work most efficiently within your organization.
If we can do that, then the workforce gap will begin to shrink, not only internationally but for each of our specific organizations. If we are training to make the team better in the aggregate, then the number of specific open jobs will start to go down. That’s my theory and I'm going to stick with it until I see some data that contradicts it.
1 Evans, K., Reeder, F., 2010. A Human Capital Crisis in Cybersecurity: A Report of the CSIS Commission on Cybersecurity for the 44th Presidency [Report]. Center for Strategic and International Studies. URL www.csis.org/analysis/human-capital-crisis-cybersecurity
2 Howard, R., 2023. Cybersecurity First Principles: A Reboot of Strategy and Tactics [Book]. Wiley.
4 Karthik, V., 2022. Moneyball for business [Website]. Linked In. URL www.linkedin.com/pulse/moneyball-business-vishwa-karthik/
6 Wright, R., 2011. Moneyball: A Look Inside Major League Baseball and the Oakland A’s [Website]. Bleacher Report. URL bleacherreport.com/articles/858470-moneyball-a-look-inside-major-league-baseball-and-the-oakland-as
7 Johnson, D., 2021. The True Story That Inspired Moneyball [News]. Grunge. URL https://www.grunge.com/677723/the-true-story-that-inspired-moneyball/
8Miller, B., 2011. Moneyball [Movie]. Amazon. URL www.amazon.com/Moneyball-Brad-Pitt/dp/B0B8SZXM1W/ref=sr_1_1
9 Chandler, G., 2022. Analytics and Instincts: What “Moneyball” Should Really Teach the Army About Data-Rich Decision-Making [Website]. Modern War Institute. URL mwi.usma.edu/analytics-and-instincts-what-moneyball-should-really-teach-the-army-about-data-rich-decision-making/
10 Rubin, S., 2020. World Series: Why Tampa Bay Rays do ‘Moneyball’ better than Oakland A’s [Website]. The Mercury News. URL www.mercurynews.com/2020/10/20/have-the-tampa-bay-rays-bested-the-oakland-as-at-their-own-game
11 Staff, 2023. Cybersecurity workforce estimate by country 2022 [Website]. Statista. URL https://www.statista.com/statistics/1172449/worldwide-cybersecurity-workforce/
13 Staff, n.d. The Workforce Framework for Cybersecurity (NICE Framework): Reference Spreadsheet [Spreadsheet]. NIST. URL https://www.nist.gov/system/files/documents/2020/11/17/supplement_nice_specialty_areas_and_work_role_ksas_and_tasks.xlsx
14 Staff, 2023. Cybersecurity First Principles Appendix [Website]. The CyberWire. URL https://firstprinciples.thecyberwire.com/CybersecurityFirstPrinciplesBook