“Stress test based on Oliver Wyman in Bank of Spain: an evaluation”

This paper, based on econometric techniques, has done a study to improve the predictions of the stress test, concerning the estimation of impairment losses. The main results obtained are: 1) the impact of the explanatory variables on the impairment loss is different at stages of growth, compared to times of recession; 2) there is a certain inertia of the dependent variable, but this inertia is different in intensity, and even the sign in the growth stages concerning the stages of recession; 3) of the explanatory variables, nominal GDP and equity are those that have a greater impact on the impairment loss; 4) finally, the two dummy variables that assess the impact of adjustment to market value of assets in the process of mergers and acquisitions that occurred in 2010, and regulatory changes implemented in 2012, have been statistically significant and with the expected signs.


Introduction
In recent years, there has been a generalized use and disclosure of the stress test. The aim is to provide security to financial markets, a sector that is significantly affected by rumors. According to Til Schuermann (2014) and Nyoka (2015), one of the consequences of the recent financial crisis is that the standard methods, such as regulatory capital ratios, are no longer reliable.
Investors need reliable tools to examine the possibility of investing in a credit institution, especially when it may be subject to impairment. In this way, they can estimate the results according to the actual risks assumed (Beltratti and Stulz, 2012). As an example of the need for this information, you can check with the significant economic consequences of this crisis. According to Conlon and Cotter (2014), the injection of public funds in failed European banks totalled 535,000 million euros in the last financial crisis.
However, stress tests have not shown that this tool is reliable. The stress test should improve in transparency issues of methodology the forecasts should be probabilistic (Basu, 2013). The lack of success in the prediction of the stress test has been shown in the case of Spanish credit institutions (CNMV, 2011; Wyman, 2012).
The three years covered by the Oliver Wyman report has been that higher impairment losses suffered by the Spanish financial system in all their history, and by far. Thus, from 2008 to 2013, the impairment losses according to the Bank of Spain in the Spanish financial system totalled 238,430 million euros and, however, from 1971 to 2007, they totalled only 76,899 million euros. In conclusion, in 37 years were only one-third compared in the latter five years (Climent and Pavía, 2015a).
Salvador Climent- Serrano, 2016. Salvador Climent-Serrano, Ph.D., Assistant doctor, University of Valencia, Spain. Therefore, to estimate forecasts, it is necessary to consider many factors, because conditions may change over time. The prediction data obtained in times of growth may not be valid for times of crisis. Not only by the change in the value of variables, of course, but also by the change in the value of the coefficients applied to the variables (Ruby and Opiela, 2015).
The aim is to propose a new model for improvement of the stress test. This new framework will consist of various econometric models. The main aspects that should be taken into consideration are the weaknesses that have been detected, especially in the Oliver Wyman report 2012 for the Bank of Spain, these are: The differential effect between periods of crisis and recession. The choice of representative variables, given that the correlation between some of the variables invalidates the econometric macromodels. The inclusion of internal variables, because the investigation has shown to have an important influence on the prediction of future results. The selected type of econometric model.
As a new contribution to the literature, ad hoc economic models for each stage are presented and internal variables of credit institutions are added to the usual macroeconomic variables.
The main results indicate that the forecasts of the Oliver Wyman report very significantly overestimate the impairment losses of the credit institutions evaluated, unlike those of the CEBS and the EBA, which very significantly underestimated impairment losses. So, the Oliver Wyman report caused an increase in capital requirements and an excess of financial panic, which led to the request of the financial bailout of up to 100,000 million euros by the Spanish state.
Another of the main results indicate that the impact of the variables is different in growth stages with respect to steps recession. A structural break occurs in the turning of the business cycle, in 2008. Inertia of the dependent variable exists, but this is different in intensity and sign in stages of growth and recession.
Results can be used to predict the evolution of the losses due to default of the credit institutions, adjusting the estimates to the stage of the economy. Moreover, with the result obtained with the elasticities, future developments in the credit institutions may be analyzed.
The rest of the article continues as follows: the first section is studying the background, and a literature review is performed, in the second, the methodology and data used are presented, the third outlines the results obtained and, in the final, conclusions are presented.

Background and literature review
The last global financial crisis and its consequences (high unemployment and slow growth) have promoted macro-prudential regulation, with the aim of reducing risks in the financial system, and social and economic costs (Covas, Rump and Zakrajšek, 2014). These costs in Spain have been very important: more than 25% of GDP, and an unemployment rate that exceeded 26%, in addition to countless families evicted from their homes, because they could not deal with their loan instalments. are of the opinion that the stress tests were not informative enough and did not give an adequate response to prevent the global financial crisis. This situation was demonstrated in the EBA and CEBS stress tests.
Despite the criticism, Petrella, Resti (2013) and Borrio et al. (2014) state that if the stress tests are designed properly, they can be very effective for good crisis management. They can also help financial stability. These authors advocate introducing the internal variables in the models econometrics as increases in credit and asset prices. In the model presented in this paper, internal variables related to the health of banks have been included.
The seminal work of Wilson (1998) stresses that all the credits must be recognized that can potentially become defaults in a certain time, in a particular economic scenario. Berkowitz (1999) proposed a methodology of stress tests with two independent prognostic distributions: one for normal conditions and one that reflects stress conditions, based on changes in an underlying factor. This approach produces two sets of projections, one of the stress tests and one of the basic model. The choice of scenarios is a matter of utmost importance, so manifest, among others: Hu, Yan, Zhao and Hua (2014), Abdymomunov and Gerlach (2014) and de Souza, Silva, Tabak and Guerra (2016).
In a work performed by Vázquez, Tabak and Soutoa (2012) with panel data in the financial sector in Brazil, main results indicate the negative relationship between NPLs and GDP growth. This study also investigated the addition of independent explanatory variables with lags, with good results. In this case also, they affect the variability of results depending on time. The authors claim that history rarely repeats itself and the circumstances surrounding the shocks are almost always different.
Schechtman and Gaglianone (2012), as well as Covas, Rump and Zakrajšek (2014) argue that quarterly data models improve the estimates. However, in the Spanish case, the model with quarterly data predictions worsens, since, according to Climent (2016), impairment losses have a marked quarterly seasonality, very pronounced in the fourth quarter, which would invalidate the model. Shechtman and Gaglianone (2012) use models with aggregate data of the credit system. The dependent variable is the default. The stress test is based on the household sector in Brazil. The main results emphasize that the unemployment rate produces a more detrimental effect, while the rate of inflation and interest rates in the most stressed scenarios show a greater impact. In this case, it is verified that certain variables may have different effects in times of growth compared to recessions. Studying these different impacts in impairment losses at the different economic stages is one of the objectives of this work. Other studies have estimated the losses of a particular portfolio, e.g., Bellotti  ). This is due to the fact that in 2010, most of these credit institutions made corporate operations, and updated their assets at market value (Climent and Pavía, 2015b), so they advanced the impairment losses, which caused a decrease in 2011. By another party to 2012, new rules were implemented causing a increase to impairment losses.

Explanatory variables.
To estimate impairment losses, we first studied the variables that have been used in the Oliver Wyman report. However, as shown in Table 1, there are strong correlations between them; this causes multicollinearity problems when estimating regressions.
The first column of Table 1 'Impairment losses' shows the correlation between the impairment losses and the rest of the variables. As mentioned in the previous paragraph regarding the problems of multicollinearity, ability to carry out regression models external variables has been significantly reduced.
There is a strong correlation between nominal GDP, the price of land, the price of homes and the unem-ployment rate. Also, between the CPI and the deflatorand between them and the GDP. Moreover, nominal GDP is the combination of real GDP and CPI, so that nominal GDP has been chosen to represent all these variables. The variables that represent the interest rates (Euribor 3M, 12M and 10-year debt), such as the exchange rate and increases in loans, are not significant in econometric models. Therefore, as external variables, nominal GDP and the index of the Madrid Stock Exchange have been chosen.
Moreover, as has been shown in the literature review, in addition to external variables, there are also important internal variables to determine the amount of impairment. The internal variables used following the authors studied in the literature are the equity and the profit rate.
Two dummy variables are also included. The first represents the change of legal regulations which substantially modified the amount of impairment losses that credit institutions should make; this variable is equal to 1 in 2012, and zero in other periods. The second is the mergers of 2010, in which credit institutions restated assets at market prices, significantly reducing impairment losses in 2011. In this case, the variable is set to 1 in 2011, and zero in other periods.  The results obtained in these few variables explain about 80% of the results, and, therefore, can be considered good predictors.

Source data.
The data for the internal variables used are taken from the annual accounts and reports of the 14 entities which underwent the stress test, including all entities previously merged from 2004-2014, a total of 68 credit institutions. The data of macroeconomic variables were obtained from the databases of INE, the Bank of Spain, the Ministry of Development and the CIS (Sociological Research Centre).
The variables used in the models are:

Impairment losses.
This is the dependent variable. It is presented as a ratio where the numerator is the sum of three accounts: 1) impairment losses on financial assets; 2) impairment losses on other assets; and 3) gains (losses) on non-current assets for sale, not classified as discontinued operations. In the denominator, the assets of the credit institution.
Equity. Ratio of equity over assets.

ROA.
Ratio of net profit relative to assets.

Econometric models.
The developed econometric models are based on unbalanced panel data. Model 1 was estimated using fixed effects after completing the homogeneity test and Hausman. These models combine cross-sections for various periods of time, so they have more data and can keep track of each individual. When using the fixed effects model, waste or errors are broken down into two parts, one part fixed constant for each individual ( i ), and another part random ( it ). This is equivalent to get a general tendency to regression, giving each credit institution a different point of origin on the vertical axis. All three models were estimated for the entire sample and for two subsamples, 2004-2008 and 2009-2014, with the aim of checking whether there is structural change between the two periods and significant differences in the coefficients.

Impairment losses
Moreover, according to data obtained from VIF (variance inflation factor), these models have managed to eliminate the problems of multicollinearity. All models were estimated using robust standard errors heteroscedasticity. Table 3 shows the results of the different models. For each variable, the first row shows the coefficient, which being a linear model represents the marginal propensity; in the second row and in brackets, robust standard errors are shown; and in the third row and in square brackets, the elasticity of the explanatory variable is shown. Source: level of significance: *** = 1%, ** = 5%, * = 10%. Robust standard errors are in brackets using White. The elasticities are between square brackets.

Results
According to the results, it appears that the impact of the explanatory variables on the impairment losses varies at different stages of growth and the recession. According to the Chow test, a structural change is produced in 2008 and the difference in coefficients between the two sub-periods is statistically significant in all the variables, except equity of the model AR (1).
The level of significance R-square corrected is quite high, especially in models that cover the entire sample, and the period of recession. One issue that is postponed for future research is to study whether the change in the explanatory variables for each of the sub-periods, can improve estimates of the first period. This means that not only the impact of the explanatory variables is different in each of the periods, but also the explanatory variables would be different.
When studying the sign of the correlation according to the Durbin-Watson statistic and coefficient AR (1), it shows that the sign changes depending on the stage. In the stages of economic growth 2004-2008, the sign is positive, that is, there is a positive inertia in the impairment losses in accordance with the previous period. However, in times of recession, 2009-2014, the opposite happens as the inertia is negative. This may be motivated by smoothing the profits, and this circumstance has already been detected in other studies such as Bikker and Metzemakers (2005), and Bouvatier, Lepetit and Strobel (2014).
When nominal GDP increases, impairment losses decrease. This relationship occurs in the full sample and the sample includes the period of recession. However, it is not significant in the growth stage. In this aspect, future research should study different variables for different stages. According to the elasticity, the effect is greater in the stages of recession with respect to growth. The impact of elasticity in the nominal GDP in the model with the full sample is 0.15%. That is, an increase of 1% in the nominal GDP translates into a decrease in impairment losses of 0.15%.
An increase in the index of the Madrid Stock Exchange decreased impairment losses on assets of credit institutions. In this case, in seven of the nine estimated models, it is statistically significant. Although the impact is very small, the elasticity is close to zero in all models.
The solvency of credit institutions, measured by the amount of equity capital, is one of the variables that has the greatest impact on the evolution of impairment losses. Also, it is statistically significant in seven of the nine models. The elasticity is around the unit, depending on the stage, being greater in times of recession relative to the growth stages. According to the results of elasticity, a credit institution in which the equity is 1% higher than the other has 1% less impairment losses, being greater in times of recession than growth. The effect of ROA has the same sign, but lower strength: elasticity is about 0.1.
The two dummy variables are also statistically significant, verifying that the impairment losses decreased in 2011 because of the upgrade market value of the assets in 2010 in almost all credit institutions, and with the positive sign in the coefficient of the dummy variable, representing the year 2012 as a result of regulatory changes that took place.

Conclusions
The stress test in recent years has become widespread as a tool to evaluate the solvency of credit institutions. However, reliability should be improved in terms of results and transparency in terms of methodology. The main results are: The R-square corrected obtained indicates that the models are good estimators/predictors of the impairment losses suffered by banks.
The impact of the explanatory variables on the impairment losses varies in different stages of growth, compared to times of recession.
There is a certain inertia of the dependent variable, but this inertia is different in intensity, and even the sign changes in the stages of growth, compared to the stages of recession.
Nominal GDP is a statistically significant variable for the impairment losses, with an elasticity less than unity. The stock market price index is also a significant variable; however, in this case, the elasticity is close to zero.
The equity is the most important variable influencing the impairment losses, with an elasticity greater than unity. The profits are also important, but with a lower elasticity.
Finally, two dummy variables have been introduced to study the impact of upgrading to market value assets in the process of mergers and acquisitions that occurred in 2010, and regulation changes implemented in 2012. The two variables are statistically significant and the results show expected signs.
In the present work, variables were standardized for the three types of subsamples. Future research, having observed these results should investigate which variables are most important in each of the stages, since it may be that for each of the three subsamples studied, coinciding with growth stage, recession, and a combination of both, that the most significant variables were different, because some of them at some stages were not significant. This way the results could be compared with those of this work, which has chosen to standardize the variables for the three samples.
Future research: the aim for future research is to study whether the change in the explanatory variables for each of the sub-periods, can improve estimates of the first period. This means that not only the impact of the explanatory variables is different in each of the periods, but also the explanatory variables would be different.

Implications of the study:
Reality has shown that the current credibility of the stress test is not good. Recent EBA stress test of 29 July 2016 published conferred an excellent level of solvency to all banks, especially Spanish. However, the impact on the stock market price on the banks of this report was a drop in the European banking sector 5.10% and 7.1% in Spain.
So, it is clear that the application of tools, such as this paper, can improve their credibility.