How is historical data being used in scenario analysis to quantify damage and better identify the impact of disruption?

Historical events are a key scenario building block. In general, we create a ‘scenario workbook’, which details material and relevant events prior to the first working session. We send this ‘homework’ out to help set a ‘frame of mind’, and for the following reasons:

1) Consensus – We identify both internal and external loss data events which are applicable and material to the business to build team support for the scenario selected.

2) Scaling – It gives scenario participants some perspective to quantify potential loss estimates and scale the data to their institution, based on actual events. For example, we used breach data costs from the ‘Ponemon Institute’ and other major cyber events as a “data points” to quantify the actual business exposure.

3) Validation – It overcomes a psychological ‘roadblock’ where people will resist the scenario as ‘unrealistic’. They are usually surprised when they find out it did happen, and in some cases, are even more surprised when a few years ago, it happened at their company.

At what point does business interruption turn into a stress scenario?

I’ve been involved in several real-life events that turned into stress scenarios, especially for the people involved. We had a major technology outage that was getting worse by the minute and impacting clients. They couldn’t pay bills or make deposits. The clients and regulators were all over us and we had to create scenarios to figure out how to resolve the client impact while still in the middle of the disruption. We ran scenarios to understand the potential for massive fraud risk versus the current technology issues. It was an exercise that created “out of the box solutions.”

What are the steps taken to identify a quantifiable stress scenario, and how do you quantify it?

It really comes down to the business the company is in and their strategic goals. At one institution, we used a filtering process that identified external losses over a certain threshold to determine if we should run that scenario. We would make a recommendation to the Risk Committee for their decision to move forward or not. For example, I was working with a retail banking client and improper foreclosures started to make the news. We quickly came together and looked at tangible data, such as their total book of mortgage business and foreclosures over the last 3 years. We made some estimates on using worst, best and expected error rates. We ran the scenario to determine if the exposure was within our appetite. Then we worked with the areas involved to do a deep dive review to prove or disprove our assumptions.

Quantification is always challenging, but a lot depends on the availability of data. If it is a brand-new condition such as Covid -19, you may be challenged to quantify without it connecting all the potential dots, which is a herculean task.

In 2017, I developed the Multiple Event Simultaneous Scenario (MESS) concept, which basically challenged people to think about multiple scenarios occurring at the same time. Prior to MESS, scenarios were largely based on a singular event, which Covid-19 taught us is not a mirror of reality. MESS can expand the businesses’ thinking and based on the total outcome, assist to quantify the total exposure and preventative actions. We’ve run MESS using up to 3 simultaneous events, which was an amazing exercise as it challenged people to make their business “bullet proof” and plan for extreme stress.

Have you been able to predict, and better respond to, any real-life scenarios with the help of quantification and scenario analysis in your experience?

The short answer is “yes.” We have used scenarios in cyber-attacks that opened our eyes to our true exposures. I worked with senior leaders from Risk, Business, Technology, and Operations to form a Scenario/Incident Management Committee. This group approved and participated in material scenario exercises and incidents. We created strong working relationships and when the scenarios turned into an actual attack, we were ready.

Based on the committee’s scenario work, we were prepared, and the attacks had very limited impact on our ability to serve our clients. Lastly, this team worked together on many scenarios and incidents which created a sense of teamwork and familiarity that made this a very highly effective and performing group.