RSA Adventures: Risk Forecasting
My Travels Down the Yellow Brick Road with Richard Seiersen and Rob Brown
At the RSA Conference at the end of April, resilience hosted a small Cyber Risk Workshop facilitated by Richard Seiersen, the Chief Risk Technology Officer at Qualys, and Rob Brown, the Senior Director of Cyber resilience. The audience was me and a handful of CISOs and other security professionals getting a deep dive on how Richard and Rob think about risk forecasting. It was the most useful and impactful session I’ve had at RSA in over 10 years.
I’ve known Richard for a while now. The Cybersecurity Canon Project inducted his book, “How to Measure Anything in Cybersecurity Risk” into he Hall of Fame back in 2018 and he and I have been discussing and debating risk forecasting ever since.
This was the first time that I met Rob but he and I immediately bonded over Fermi Estimates and fanboyed over the man they are named after, Dr. Enrico Fermi. Fermi played an important part on the Manhattan Project; the development of the Atomic Bomb at Los Alamos during WWII. And if you knew what you were looking for, he popped up in last summer’s blockbuster movie, Oppenheimer.
Richard Seiersen’s Dead Guy Quotes
One thing I love about Richard is that he likes to quote dead guys to make his point These are some of my favorites:
As far as the laws of mathematics refer to reality, they are not certain; and as far as they are certain, they do not refer to reality. –Albert Einstein
Although this may seem a paradox, all exact science is based on the idea of approximation. If a man tells you he knows a thing exactly, then you can be safe in inferring that you are speaking to an inexact man. –Bertrand Russell
Tactics without strategy is the noise before defeat. –Sun Tzu
Strategy Is The Economy Of Forces –Clausewitz
All of these line up nicely with my own book on cybersecurity first principles. A thing I had to learn while writing it was that just because “math” is involved, calculating risk does not result in five-nines of precision and accuracy. As Russel says, it’s an approximation. It’s a measure of uncertainty. But, with a few Superforecasting tricks and advice from Seiersen and Brown, the answers are good enough to make resource decisions with.
Another thing I have noticed when talking to senior infosec professionals, most don’t understand the difference between strategy (the what we are trying to do) and tactics (the how we are going to pursue the strategy). As Sun Tzu says, that leads to disaster.
In my book, I make the case that the ultimate cybersecurity strategy, the absolute first principle, is to reduce the probability of material impact due to a cyber event in the next say one to five years. That is the “what” we all should be trying to achieve. “How” we do it, the tactics, is different for every organization and is based on available resources, organizational culture, tolerance to risk.
Materiality = Insurance + Cash on Hand
Richard and Rob covered a lot of fantastic material. Like I said, it was a great learning afternoon. But one thing in particular jumped out at me and I’ve been thinking about it ever since. They displayed a typical loss exceedance curve that you might see for any company. It showed projected losses for a range of probabilities. They suggested that the projected losses, the curve so to speak, should be below whatever cash the company has on hand for emergencies plus the cash expected from the insurance policy. My mind immediately thought that this was the top end of the very definition of materiality.
Ever since the U.S. Securities and Exchange Commission (SEC) made the ruling back in July of 2023 requiring public companies to report material cybersecurity incidents within five days of discovery, infosec professionals have been trying to understand just exactly what materiality means. Before the ruling, determining materiality was not something that CISOs traditionally got involved in. After, we still are not the decision maker about what is and what isn’t material. That’s the CEOs and CFOs job, but we now have input to the calculation.
The first step then, in understanding what materiality means to your business, is to talk to the CFO and the CEO. That said, I’ve been trying to get my head around the concept. I do know that the materiality number is a range of losses between something that will significantly hurt the company on one end and a company killer on the other.
And while saying “company killer” in a room full of CISOs gets their attention, its still an abstract idea. Without mentioning materiality, I think Richard and Rob put their finger on the exact company killer number for every organization. In order to survive a material cyber event, to pay for the total cost of the incident and still keep the business running, to ensure that the business continues during and after the incident, the loss has to be below whatever cash the company has in reserve plus whatever they might receive from insurance.
Mind blown!
Great by Choice
Serendipitously, just before the RSA Conference, I stumbled upon on a 10-year-old review I wrote on a Jim Collins book called “Great by Choice: Uncertainty, Chaos, and Luck—Why Some Thrive Despite Them All.” Collins and his research partners have been studying commercial company performance for the better part of three decades and have published several books based on their research. Many of them were NYTs Bestsellers.
“Great by Choice” focuses on seven pairs of similar companies between 1972 and 2002. Half of them, Collins calls them the 10Xers, cumulatively outperformed their comparison companies in terms of stock returns by a factor of 35%. The point of the book is to determine why some companies wildly succeeded in a time of chaos and upheaval while other companies who experienced the same or similar situation did not.
One trait that Collin’s highlights is “Productive Paranoia.” 10Xers learned how to effectively manage risk so that the organization was never in mortal danger. He said that building cash reserves or contingency plans to weather unexpected downturns or crises allows them to survive shocks that might destroy less prepared competitors.
Materiality = Insurance + Cash on Hand
I think Richard, Rob, and Jim are all talking about the same thing.
References
Bob Clark, Douglas Hubbard, Richard Seiersen , 2017. Author Interview: “How To Measure Anything in Cybersecurity Risk” - Cybersecurity Canon 2017 [Video]. YouTube. URL
Christopher Nolan, Cillian Murphy, Emily Blunt, Matt Damon, Danny Deferrari, as as Enrico Fermi, 2023. Oppenheimer [Movie]. IMDb. URL https://www.imdb.com/title/tt15398776/
Douglas Hubbard, Richard Seiersen, 2016. How to Measure Anything in Cybersecurity Risk [Book]. Goodreads. URL https://www.goodreads.com/book/show/26518108-how-to-measure-anything-in-cybersecurity-risk
Erik Gerding, 2023. Cybersecurity Disclosure [Explainer]. US Securities and Exchange Commission. URL https://www.sec.gov/news/statement/gerding-cybersecurity-disclosure-20231214
Jay Jacobs, 2017. Communicating Risk: Loss Exceedance Curves [Explainer]. Cyentia Institute. URL https://www.cyentia.com/communicating-risk-loss-exceedance-curves/
Jim Collins, Morten T. Hansen, 2011. Great by Choice: Uncertainty, Chaos, and Luck—Why Some Thrive Despite Them All [Book]. Goodreads. URL https://www.goodreads.com/book/show/12675109-great-by-choice
Richard Seiersen, Robert Brown, 2025al. Cyber Risk Workshop, San Francisco [Seminar]. Qualys / resilience. URL https://www.qualys.com/lp/tp1/cyber-risk-workshop-san-francisco-april-30-2025/
Staff, 2025. Home Page [Company Shingle]. Resilience. URL https://cyberresilience.com/
Steve Winterfeld, Rick Howard, 2017. Book Review: How to Measure Anything in Cybersecurity Risk [Book Review]. The Cybersecurity Canon Project. URL: https://cybercanon.org/how-to-measure-anything-in-cybersecurity-risk/