ECB working paper: A false sense of security in applying handpicked equations for stress test purposes

07 September 2015

The paper promotes the use of Bayesian model averaging for the design of satellite models for stress testing. Banks employing ’handpicked’ equations risk significantly underestimating the response of risk parameters and overestimating their capital absorption capacity.

Stress testing has become a very conventional and increasingly prominent tool for assessing the resilience of financial institutions to hypothetical macro-financial stress scenarios. One significant recent stress test assessment (including as well as an Asset Quality Review) was conducted by the European Central Bank in the course of 2014 for 130 significant European banking groups that are now under the direct supervision of the Single Supervisory Mechanism (SSM).

This paper aims to address one important element that all stress tests involve — whether conducted by financial institutions themselves (in a bottom-up fashion) or by some central authorities (in a top-down fashion) — which lays in the use of satellite equation systems for translating macro-financial shock scenarios into risk parameters at bank level. The concern that forms the basis for this paper is the fact that virtually all institutions tend to neglect the presence of model uncertainty. While the bridge equations for a given risk parameter at bank-level may be sound and look acceptable from an economic and econometric viewpoint, and therefore pass an internal risk management or supervisory sign-off, there is a risk that the chosen specification by the institution would underestimate the risk parameter response and in the sequel overestimate the loss absorption capacity of the bank.  The choice of equations that result in overoptimistic scenario conditional forecasts might either be due to explicit incentives for banks to underestimate the cost of risk or be coincidental.

The aim of this paper is to promote the use of a Bayesian model averaging (BMA) methodology to mitigate that risk. The model averaging philosophy is not new and used in other areas by researchers and econometric practitioners. With the BMA-based models and the illustrative stress test simulation results for a sample of 108 SSM banks authors are aiming to make a simple point: that the deviation with regard to the banks’ projected capital position can be very significant when either employing some overoptimistic handpicked satellite equations or, as authors argue, the more robust BMA-based satellite models. Handpicked equations may pass a set of basic criteria for economic and econometric soundness, while implying however unduly benign risk parameter responses to an assumed adverse scenario. They therefore pose a risk for an institution to be significantly under-provisioned.

Supervisors, as well as the institutions that are being supervised, may consider using this approach in order for a risk assessment across portfolios to be more robust, i.e. more likely reflect the relative risks of exposures to different regions and segments. Moreover, it shall help develop a more level playing field also across banks, with portfolios of similar (say equal, hypothetically) risk characteristics more likely resulting in similar capital requirements, if conditioned on the same centrally-defined macro scenario.

Full working paper


© ECB - European Central Bank