meets the “empirically derived, demonstrably and statistically
sound” standard and other requirements of Regulation B. More
broadly, it is important to evaluate the representativeness of the
data used to develop a model, and whether any of the variables/
attributes (or combinations of variables) in the model are likely
to have strong correlations with a prohibited basis or otherwise
might be controversial.
It is also important to evaluate the relevance of the variables
in a model to the behavior or outcome the model is designed to
predict. A consumer’s past credit performance or current financial
situation have direct intuitive relationships to future credit performance. However, data elements that appear to have predictive
power but have no intuitive relationship to the credit behavior
being predicted, should receive extra scrutiny. Such elements can
be challenging to defend in the event that they create a disparate
impact, and also can be challenging to explain to consumers in
terms of adverse action reasons if they result in a denial decision.
When potentially risky or questionable variables are encountered,
it is important to evaluate how much they actually contribute to
the predictive power and business objectives of the model, and to
weigh those benefits against fair lending risk in deciding whether
the variables should be used. A variety of statistical tools can be
useful in evaluating the tradeoffs.
Second, it is important to ensure that models receive a rigorous statistical validation by a qualified, independent internal or
external party to ensure they model are statistically sound and were
developed according to generally accepted statistical methods.
Statistical validity is an important line of defense against potential disparate disparate impact claims. If a model or
decision variable is found to have a disparate impact
on a prohibited basis, it may still be permissible
(i.e., not illegal discrimination) if its use is
supported by a sufficient business justification. Statistical validation is aimed
in part at confirming the evidence of
that justification.
Third, it is important to ensure
that a model’s performance is regularly monitored over time. If
the predictive power of a model
quickly degrades over time,
that is a sign the model might
not have been statistically
valid to begin with or that
the correlations on which the
model was originally based
may have been idiosyncratic
to the particular data sample
or time period used to develop
the model.
Fourth, it is important to
identify and address any circumstances where the decisions
of the automated system may be
overridden and where there may be
human touch-points in the process.
For most of the marketplace lending processes the authors have
observed, exceptions are very rare or non-existent, and the only
human interaction in a loan application process might be in
reviewing potential fraud risks and in requests for additional
information from the applicant. Where there are opportunities
for human intervention in the decision process, controls testing,
quality control and statistical analysis can be used to diagnose
the potential for fair lending risk.
Finally, retention of relevant documentation and data is critical
to managing fair lending regulatory risk. Traditional banks know
the importance of full documentation and retention, but this level
of attention may not yet be in place for marketplace lenders. The
data used to develop a model and documentation of the model
development and validation processes should be retained because
they are likely to be needed to perform fair lending testing and
in the event it is necessary to defend against a discrimination
claim. The data used in each credit decision should be retained for
the same reasons. If application data is updated and overwritten
over time (as sometimes occurs when the lender has an ongoing
lending relationship with the consumer) it may be impossible to
confirm in a retrospective review why a consumer was approved
or denied, and thus to defend all of the credit decisions made.
Marketplace lending, machine learning, and Big Data offer
important benefits to both those who supply and those who
demand credit. For stable and responsible growth in this area
to continue, responsible lenders will couple strong fair lending
compliance oversight with their emphasis on revenue generation
and credit risk management. ■
ABOUT THE AUTHORS
MARSHA COURCHANE, Ph.D., is a vice president and co-Practice
Leader of Financial Economics in the Washington, D.C. office
of Charles River Associates, an economic consulting firm. She
specializes in financial institution analyses for regulatory reviews
and in support of litigation. Dr. Courchane engages in research
and analyses with respect to mortgage markets, discrimination in
lending, consumer credit, securitization, credit risk, and redlining
issues. Dr. Courchane previously served as Director of Financial
Strategy and Policy Analysis, Housing Economics and Financial
Research at Freddie Mac, and as a Senior Financial Economist in
the Risk Analysis Division Office of the Comptroller of the Currency.
She can be reached at mcourchane@crai.com.
DAVID SKANDERSON, Ph.D., is a vice president in the Financial
Economics Practice in the Washington, D.C. office of Charles
River Associates. He spends his workdays pouring through
data and estimating complex statistical models to help lenders
assess, monitor and manage their fair lending risk, including
the evaluation of credit scoring systems. He also serves as an
expert in mortgage litigation matters. Dr. Skanderson previously
led departments responsible for fair lending analysis, HMDA
compliance, and compliance loan review at Washington Mutual
Bank. He can be reached at dskanderson@crai.com.
The views and opinions expressed herein are those of the authors and
do not reflect or represent the views of Charles River Associates or any
of the organizations with which the authors are affiliated.
