Given the possibility that the optimized version of static balance sheet simulation could run significantly faster than existing production balance sheet simulators, how would you best use the faster code to automatically discover errors or inconsistencies in the production code outputs? Let’s assume GSIBs report their balance sheets simulated against ten market/econometric scenarios twice a year, given a new set of balance sheet simulations and the previous two sets of simulations, from six months and a year ago what tests would you like to run to verify the consistency and accuracy of the new runs? You can run these tests from the perspective of internal audit, where you have access to the the product models and can rerun the balance sheet simulation with the internal models. On the other hand, you can run another set of tests from the FRB perspective where you do not have explicit access to the product models but you know the generic product types, and you know all the other bank’s balance sheet submissions for the previous two semiannual reporting cycles in addition the simulations for the current cycle.
What types of errors would you look for:
- Are all the accrual positions covered in all the scenarios?
- Are the positions consistent through time?
- Are the positions modeled consistently over time?
- Are there obvious model changes?
- Are positions modeled to the full simulation horizon?
- Are the market data dependencies consistent over time?
- Is the market data consistent in each of the portfolio simulations?
- How many positions are modeled hold flat is that consistent over time?
- Is the simulation reversible – can I run it backwards and recover the original balance sheet?
- How much of the simulation is overridden, are the overrides consistent?
- Can you identify dead market data – market data that is not used in any product model simulation?
- Can you identify dead positions – positions that do not move with any market data?
- Can you identify dead econometric data relative to the simulation?
I’m sure that I will think of more and add to this list.
The advantage you have with a fast balance sheet simulation check is that you can rerun a thousand perturbed markets in about a minute then do arithmetic on the simulator output. So you could move each of 1000 pieces market data and find out what the dependencies are. Moreover some of the differences you observe should be explainable in terms of the market moves and sensitivities reported. You could write the explanatories program for the accrual portfolio over each reporting period. Similarly you can move econometric data and you can perturb the balances an rates themselves. It is easier to do if you are Internal Audit and have access to the actual product models. But you are not without options as the external FRB examiner as well.
Need to think this through.
I think there are results to be had here.