Back Testing – What’s the Big Deal?

Beneath the wonderful world of interest rate risk (IRR) and asset liability management (ALM) lies a foundation that must be built to ensure an effective and reasonable IRR process. A large part of this foundation is developing an in-house independent review of the IRR system. The FDIC, along with other federal and state banking agencies, has repeatedly stressed the importance of an independent review for the IRR process. The FDIC defines independent as “relying on internal audit staff, bank employees independent of the IRR management process, or third-party consultants. Importantly, there is no requirement or expectation for a bank to hire a consultant, and most community banks should be able to identify an existing qualified employee or board member to periodically conduct this review.”

The 1996 Joint Agency Policy Statement on Interest Rate Risk lists five elements of an independent IRR review:

1. The adequacy of, and personnel’s compliance with, the bank’s internal control system.
2. The appropriateness of the bank’s risk measurement system given the nature, scope, and complexity of the bank’s activities.
3. The accuracy and completeness of the data inputs in the bank’s risk measurement system.
4. The reasonableness and validity of scenarios used in the risk measurement system.
5. The validity of the risk measurement calculations.

Validating, or back testing, IRR model results is a crucial part of an independent review and directly hits on three out of the five elements listed above. The back testing analysis is focused on earnings-at-risk and shows the impact of many underlying assumptions, specifically repricing balances and rates of key products within the IRR model and given rate environment. When a back test is performed, it compares the IRR model results and projections against actual performance from the institution for a one-year period. To validate the assumptions and data inputs used within the model, a back test will often provide a percentage variance between the model’s calculations for projected net interest income versus actual performance. The larger the variance, the more questions that need to be answered. What we often see is a variance range, or threshold, that institutions try to stay within for the variance to be considered “acceptable.” However, even if a back test is within the acceptable range, it is still important to understand what is driving these variances. Focusing solely on an overall variance range and not the variance drivers themselves can lead to flawed model assumptions and outputs. It is recommended to perform a back test at least annually.

Variances in back testing are often driven by balance sheet composition changes and rate changes. An example of this type of variance can be seen over the most recent rate cycle. As the federal funds rate rose over 500 basis points from March of 2022 to July of 2023, the average institution saw a large shift in deposits moving from non-maturity deposits into higher paying CDs and borrowed funds. This large volume movement and aggressive rise in rates resulted in significant variances in back tests from March of 2022 to today.

So, what does this variance truly tell us? Why is it important and what is the big deal with this variance? Let’s take the large variances that we saw in relation to CDs as an example. These variances would suggest that it may be appropriate to review rising rate shift sensitivities (Betas) on these products. Accurate assumptions and inputs result in accurate outputs. If we ignore variances from a back test and do not make proper adjustments to assumptions, the results from the IRR model can become skewed and misleading. It is important to note that one should not wait for a large variance on a back test to adjust assumptions. The process should be proactive and ongoing. Back testing allows us to validate the assumption development process and make additional adjustments if needed. As the Federal Reserve gears up for rate cuts, it will be critical to check falling rate assumptions and ensure they are modeled realistically and properly. Continuing with the same example, we saw CD rates skyrocket as the federal funds rate rose. But will we see that same impact as the federal funds rate falls? These are questions we should ask, and then use the back test to help answer these questions.

To reiterate, independent reviews are an important aspect within the ALM and IRR process. Back testing helps to affirm a good process by validating model assumptions and assists in making improvements to model inputs when necessary. And while the variance range from a back test is important, it is also important to not solely focus on that one output. Instead, it is crucial to understand the reason, or lack thereof, for the variance in the first place. As we continue through this rocky rate cycle, IRR and back testing will continue to take a front seat in helping institutions make important decisions.