Some key regulators and bank lobbyists characterize much of bank supervision as checking the box. “Checking the box” suggests a mindless emphasis on the superficial. Maybe the phrase test-markets well. But is box checking really that prevalent? My own experience suggests that it isn’t and that there are some fundamental contradictions associated with this narrative. Moreover, proposed changes to bank supervision are likely to make box-checking more common, not less.
The On Message Metaphor
The Bank Policy Institute (BPI), the lobbying arm for the country’s largest banks, has led the charge in use of the box-checking metaphor to characterize aspects of bank supervision they dislike. The term “check-the-box” appears 9 times on the BPI website. “Box-checking” appears 10 times. We’ve seen similar statements from some of the leading bank regulators. Treasury Secretary Scott Bessent has stated that agency “leadership must drive a culture that focuses on material risk taking, rather than box check checking.” Acting FDIC Chair Travis Hill has decried checking “process-related boxes.” Fed Governor Michelle Bowman expressed concern “that overemphasis on process and supervisory box-checking can be a distraction from the core purpose of supervision.”
Well, they certainly are on message. At least Hill and Bowman have described box checking as something to be avoided. Hill has even acknowledged that it’s easier to check boxes than to second-guess bank management. Comptroller of the Currency Jonathan Gould has gone a step further. In remarks before the Financial Stability Oversight Council, Gould made a much less equivocal statement:
“Following the 2008 financial crisis, we implemented a regulatory and supervisory framework that sought to micromanage bank balance sheets and activities, and to reduce much of the examination process to a compliance exercise focused on procedural box-checking.”
How the hell should he know? Gould previously spent a little over two years as OCC’s Chief Counsel. That’s a very important position, but one that has little to do with day-to-day bank supervision. He might sign off on enforcement actions and new regulations. Perhaps he’s seen a supervisory letter or two, but that would be about it. OCC’s legal staff rarely participate in exams, though some have moved between legal and supervisory roles during their careers. But Gould essentially started at the top at the OCC. His previous experience was as a Congressional staffer and at consulting and law firms. Most of what he would know about bank supervision is from what his clients told him.
My Perspective
Bank supervision usually occurs beyond public view. Some things, like formal enforcement actions and examiner handbooks are public information. Material loss reviews include historical data on exam ratings and other supervisory actions if a bank fails. But most of the rest happens behind closed doors. This can lead to an unreliable narrator problem, where a bank can claim unfair or unreasonable treatment without having to provide any real evidence. I spent more than 35 years in various bank supervision roles and can share my own experience. While my perspective is hardly definitive, it is likely supported by stronger evidence than those held by lobbyists or political appointees. I’ll also point out some key contradictions with the current narrative.
Most of my career focused on safety and soundness. Box checking might be more prevalent in compliance examinations where it would make more sense. Determining whether a bank fulfills certain requirements of a law or regulation is an important starting point. If your experience focused primarily on compliance, please share your thoughts in the comments.
OCC’s supervisory process for larger banks usually consisted of a series of target reviews over the course of the year. We spent the rest of our time on “ongoing supervision,” which typically involved reviewing the bank’s MIS, especially its risk reporting, and meeting with bank management. We would also follow up on previous findings.
Target reviews start with a set of broad objectives, usually amounting to a handful of bullet points. Examiners have some latitude in how to address these objectives, which might depend on the experience and skill set of the exam team. We rarely used checklists. Examiner handbooks provided a useful reference point, but not something we usually relied on procedure by procedure. We tried to take a holistic view of the bank’s risks and controls, sometimes using experience at other banks as a basis for comparison. The resulting supervisory letter to bank management summarized our findings and identified any corrective actions. A more thorough description of supervisory letters is provided here.
Some activities required more box-checking. OCC instituted a set of procedures to address derivatives activity, following the London Whale episode. The procedures themselves include the key components of risk management for end-user derivatives. However, their vast scope can require a fair amount of box checking to complete the review in a timely manner. We might spot some glaring gaps in risk management, but there usually wasn’t much depth to the reviews. The review was mainly conducted offsite and MRAs were rare. (I’m not aware of any.) While these reviews added some value, there is also a good case for cutting them back on efficiency grounds. The problem is that light touch, somewhat superficial reviews aren’t what banks have been complaining about.
Model Risk Reviews
Consider model risk reviews. These can vary from standalone reviews of model development and model validation functions to part of a wider review of a business or risk area. BPI has described model risk management guidance as “a set of check-the-box instructions on how banks should validate and document the models they use.” Oh, really? The guidance provides very broad model risk management principles that reflect some lessons from the Global Financial Crisis. For example, a model validation should assess “the quality of model design and construction.” The guidance also recommends monitoring that “confirms that the model is appropriately implemented and is being used and is performing as intended.” Validations should also compare “model outputs with corresponding actual outcomes.” In other words, model validations should try to determine whether a model actually works. None of these principles should be even remotely controversial. There is nary a checkbox in sight.
If model risk reviews were truly box-checking exercises, banks would have it easy. Examiners would need only to confirm that the bank has a policy and conducted validations. Those are necessary but insufficient elements of such a review. Actual model risk reviews proceed differently. Prior to the exam, examiners get an inventory of existing models for a particular business or risk area. They then select a sample of models. OCC would often bring in experts with advanced degrees to review the models. Those experts go through model development and validation documents with a fine-tooth comb. Examiners would then grill model developers and validators in a series of meetings.
The review would then try to determine three basic things. First, did model developers adequately support model assumptions and other modeling choices? Second, is the model accurate? Third, does the model validation function provide effective challenge? Falling short in any of these areas can result in an MRA. There is a fair amount of judgment involved, and reasonable people can disagree on both general and specific conclusions. But it’s hardly a check-the-box exercise.
Physician, Heal Thyself
Mr. Gould has accused supervisors of engaging in both micromanagement and box-checking. Ms. Bowman claims the existing supervisory framework relied on too much examiner judgment. The Federal Reserve has gone so far as to make the Orwellian proposal that holding companies can be rated “deficient” in capital planning, liquidity, or governance and controls could still be deemed “well managed.” The suggestion seems to be that examiners are both mindlessly checking the box and using too much judgment. It reminds me of a line from the song Industrial Disease by the band Dire Straits: “Two men say they’re Jesus, one of them must be wrong.”
Protestations to the contrary, incoming regulators seem to be favoring box checking over any real analysis. Revised approaches to resolution and recovery planning provide a good case in point. The FDIC recently revised its approach to evaluating resolution planning by stating, “FDIC staff expects to focus its review on the quality and thoroughness of the submission, rather than a comprehensive verification of capabilities or evaluation of projections. A submission that is responsive to each of the content requirements would be unlikely to result in material weaknesses or significant findings.” Although unrealistic resolution plans made the closures of SVB and Signature Bank more haphazard, the FDIC will only look to see if the bank completed the forms. In other words, they will substitute box-checking for any real analysis.
One way to avoid both box-checking and examiner judgment is to assume the problem away. OCC is trying this approach for recovery planning. OCC intends to rescind guidance that requires banks over $100 billion to have recovery plans. In explaining this change, OCC begins with a reasonable statement grounded in reality: “impact assessments for such recovery options appear conjectural by nature and of limited utility in an actual stress scenario.” The agency follows that statement with a whopper: “covered banks are well attuned to indicia of stress without regard to the presence of the recovery planning triggers and escalation procedure expectations of the Guidelines.” Well only if you disregard what happened in 2023, not to mention in 2008. But at least the statement is on message.