ABA Bank Compliance - July/August 2014

usage generally does not have immediate and observable life or death consequences, but an unsound or badly constructed financial model can create billions of dollars of direct and collateral damage and can severely harm the lives of customers, employees, shareholders, and their dependents.

For most financial models built on mathematical rather than subjective inputs, validation can be summarized in answering three questions.

1. Is the math right? This can be answered by inexperienced, yet technically proficient analysts. The answer says little about conceptual soundness and more about the implementation, e.g., are the algorithms, operations, and calculations correct? Are they coded correctly and performed in the right order? These questions are difficult to answer for blackbox vendor models.

2. Is it the right math? This requires more than technical skills to answer. Knowledge of competing theories and methods, the operating environment and business experience, and common sense and judgment are required as well. The answer requires the validator to understand the context of use and to confidently assess and criticize the eight development steps listed above, which together provide a sequence of calculations designed to represent a real business, social, or economic phenomenon. It also requires communication skills, including listening skills to assess whether developers are doing and understanding what they say they are doing, and that is true regardless of where the model was built.

This combination of expertise, skills, and experience is rather rare, in high demand, and, therefore, expensive.

3. How is the model governed? This includes assessing controls like change management, documentation, processing, and ongoing monitoring of use and performance.

For any particular model, a strong and knowledgeable validation department can identify all design weaknesses, limitations, uncertainties, implementation errors, and governance programs. However, even the best group cannot effectively reduce a firm’s model risk if it is improperly oriented.

That’s why all models within the firm must be:

1. Identified on a central inventory along with all relevant characteristics.

2. Classified (or ranked) according their potential for adverse consequences.

Many banks use a two-dimensional classification scheme, where one dimension is size, measured in the financial effect of the model’s results or importance. This is more general and includes financial measures as well as other factors like the intensity of regulatory scrutiny and possibility of reputational risk.

For the second dimension, some banks use complexity, which implies an assessment of the implementation in, say, SAS or Excel. Complexity may be measured by the lines of code or the number of calculations, whereas other banks use uncertainty, which hints at epistemic risk or environmental volatility. We prefer “ uneasiness” as the second dimension; it captures aspects of epistemic risk that well specified, empirically-supported theories may not

The ABA Reference Guide to Regulatory Compliance, 24th Edition is a recommended study resource for individuals preparing for the Certified Regulatory Compliance Manager (CRCM) Exam, and is an ideal resource for compliance managers.

Revisions in the 24th Edition include the CFPB’s new mortgage rules including the TILA/RESPA disclosure integration rules; the Protecting Tenants at Foreclosure Act; significant updates to the Municipal Securities Rulemaking Board rules; and, the FACT Act rule updates.