Bank and credit union executives today are accustomed to sitting through artificial intelligence pitches from a wide range of vendors. The pitches are confident. The demos look good. But ask how any of it accounts for a first-day letter, a concentration limit or an exam finding, and the answers run out fast.
Some of what’s pitched is honest about being a general tool: a horizontal AI platform your team can point at banking work the same way they’d point it at anything else. That’s a fair pitch, and plenty of institutions get real value from it. The pitches to look at more closely come from vendors claiming to offer banking-native AI.
If management teams ask what that means, most answers describe something added after the fact: banking documents fed into a general model, a fine-tuning pass, a chat interface with your logo on it. That changes what the model has read, but it doesn’t change how well it reasons. A system that has absorbed banking vocabulary can talk about banking. Whether it can be trusted with a judgment call, a lending exception or a compliance gray area is a different question.
As the vendor landscape expands to include foreign-developed models with unclear data practices and opaque ownership structures, the scrutiny required only increases.
Management teams should focus on assessing whether the system will compound into value or risk. Asking the following three questions will get your institution past the pitch and toward an answer.
What was it trained on, and can it learn your institution?
Regulations describe requirements in general terms. But enforcement actions, examination findings and supervisory correspondence show how examiners apply those requirements to specific, messy cases. Two banks can have functionally identical underwriting policies and still get different examiner reactions because of a detail an enforcement action revealed that the regulation itself doesn’t mention. A model trained only on the text of the rules is unaware of this context. Management should ask vendors to name the actual document types their model was trained on, not just the topic areas.
But industrywide training only gets you so far. Every financial institution also carries unique context no industry dataset contains. That can include policy exceptions, findings from the last exam, the reasoning a credit committee used the last three times a similar deal came up or the unwritten judgment calls the best underwriters make. A system that can only reason from general banking knowledge will treat your institution like every other one it’s seen.
A better-designed platform encodes this institutional knowledge into a knowledge graph that maps the relationships between the institution’s own policies, regulatory obligations and operational procedures.
Does it know which regulatory framework applies, and does it know when that framework may have changed?
Guidance gets rescinded and replaced more often than most institutions track closely. For example, SR 11-7, the model risk management guidance banks relied on for over a decade, was rescinded in April 2026. In its place, the Federal Reserve issued SR 26-2 as part of a coordinated interagency update that included parallel guidance from the Office of the Comptroller of the Currency and Federal Deposit Insurance Corp. A model trained on a static snapshot of the rules has no way to know that happened unless someone manually updates it. Until they do, it will keep citing guidance that no longer applies.
Was banking part of the design from the start or added later?
Two documents using similar language aren’t always subject to the same rules. A model that pattern-matches on word similarity can sound right and be wrong in exactly the cases where the stakes are highest, such as an edge-case lending decision or a compliance question with no clean precedent.
Career bankers build their judgment over years of watching where the rule and the reality diverge. A banking knowledge graph encodes that judgment structurally. It considers not just what the rules say, but how they relate to each other, which takes precedence when they conflict and where institutional policy is more restrictive than the underlying regulation.
Not every vendor in front of the management team today will have a good answer to all three questions. That is fine. Most institutions are still early in this evaluation. What matters is asking them now. The tool your institution approves today will keep applying the same kind of reasoning for years to come. It is better to find out today what that reasoning actually looks like, so you can determine whether your AI strategy will compound into value or risk.
Previous