Customizing a generative artificial intelligence model to suit your specific needs and running it at the cost of electricity probably sounds too good to be true for most financial institutions. While open weight AI models offer both of those possibilities to banks and credit unions seeking affordable alternatives to increasingly expensive commercial options, they also present certain challenges and risks. Some of those are even being debated at the highest levels of government.
Weights, in this context, are numerical parameters that ultimately shape how an AI model processes information, according to Stanford University’s Institute for Human-Centered Artificial Intelligence. In each model, the weights are continuously adjusted during training to minimize errors and improve its performance. The final, learned weights represent the model’s knowledge base.
Commercial large language models (LLMs) like OpenAI’s ChatGPT or Anthropic’s Claude are closed models, which means the weights that make up their knowledge base are not publicly available. But in an open weight model, the weights are public and anyone can download the entire model for free. Once downloaded, an open weight AI model can run a local computer.
Nicholas Merizzi, AI infrastructure leader for Deloitte Consulting, says anyone with the right technological know-how can then modify the weights as well. “You have access to change, fine tune and train them on your data to best suit your needs for the enterprise,” he says.
Not As Easy As It Sounds
To be sure, simply downloading an open weight AI model, fine-tuning the weights to suit your business needs and running it locally will only get you so far.
Merizzi says a financial institution wanting to get the most out of an open weight AI model must have access to a graphics processing unit (GPU). A GPU can quickly perform the enormous number of mathematical calculations needed to produce original content — such as text, images, video, audio or code — in response to a user’s prompt or request. A bank or credit union can own and operate its own GPU, or access one through a cloud computing platform such as Microsoft Corp.’s Azure or Amazon Web Services.
“You marry that with an open model, and then you can get the right accuracy with the right latency and the right performance,” Merizzi says. “And that becomes a very viable exercise for smaller businesses to probably explore.”
Will Rhoads, chief information officer (CIO) at Sonata Bank in Brentwood, Tennessee, is making plans to deploy open weight models for employees with lower-priority AI use cases. The $282 million institution specializes in banking franchise restaurants and providing sponsor banking services to fintechs.
“We’re building out our own interface that we’re going to serve to the employees, versus letting them go directly to Claude or directly to ChatGPT or [Microsoft] Copilot,” he says. “And we know that those are not going to be able to compete with [Claude] Fable, but how did we feel about the models that we were using six months ago? Were those not powerful models? Were those not transformative models? Well, we can run those models for the cost of electricity.”
Lamont Black, founder of Wide Open Ventures, which provides AI consulting services to small financial institutions, says tech-savvy CIOs can definitely make use of open weight models — but only if they commit to staying ahead of the curve. “It can be hard to make sure that it’s staying up to date and at the frontier,” he says, because automatic software updates are not available for locally-hosted models. “And as people become aware of what those larger LLMs are capable of, I think they’re going to get more and more frustrated with AI that can’t do that. So, that might be a solution, but I think those gaps are going to become more felt.”
But Merizzi believes AI providers will continue to release open weight models with greater capabilities that are also easier to implement. “[I’m] not belittling the point that you need some engineering chops to be able to do it, but I do think that there’s enough in flight right now where. . .The on-ramp will become smoother,” he says.
Risks and Controversies
The benefits of open weight AI models do come with certain risks. Cisco Systems, a global networking and cybersecurity leader, published a 2025 analysis of eight open-weight AI model vulnerabilities and found they are specifically susceptible to “multi-turn, jailbreak attacks.” Cisco found these cyberattacks, which attempt to solicit information through a number of indirect prompts instead of asking one direct question a model is likely to deny, were 93% effective.
The sharp rise in this “underscores the lack of mechanisms within models to maintain and enforce safety and security guardrails across longer dialogues,” the report states. “To counter the risk of adopting or deploying unsafe or insecure models, organizations must consider adopting advanced AI security solutions.” That risk further adds to the complexity of managing and securing such models.
And within the last few months, there have also been growing concerns in Washington, D.C., that too many open weight AI models are now coming from China. While American companies like Meta Platforms, Alphabet’s Google and Microsoft all offer open weight AI models, Chinese companies such as Hangzhou DeepSeek Artificial Intelligence Co. and Beijing Moonshot AI Technology Co. have ramped up their offerings to a degree that has caught the attention of both the White House and Congress.
The Wall Street Journal on Aug. 4 reported that President Donald Trump’s administration was considering imposing restrictions on Chinese open weight models. But Bloomberg on that same day reported the White House ultimately decided to exempt all open weight models from undergoing government cybersecurity safety testing to which many closed, frontier AI models like Claude Mythos are being asked to submit. The Trump administration has not formally addressed those reports and did not respond to FinXTech’s request for comment on the matter.
Sen. Jim Banks, a Republican from Indiana who sits on the Senate Committee on Banking, Housing and Urban Affairs, pressed the issue further in an Aug. 13 letter sent to Christopher Phelan, chair of the White House Council of Economic Advisers. Banks encouraged Phelan and other members of the Trump administration to offer incentives for the production of more U.S. open weight AI models in order to counter China’s efforts.
“The Chinese pursuit of open weight dominance should come as no surprise. The Chinese government has long pursued exactly this kind of global market capture using cheap, oftentimes illicitly acquired technology,” Banks wrote. “The United States must think creatively about the types of incentives that may change this calculation and promote a strong American open source ecosystem.”
Seth Winter, a partner at the law firm Troutman Pepper Locke, says those recent developments offer a good reminder for financial institutions to consider all possible risks when looking at these models. “Consider a bank that switches to an appropriately secure Chinese open-weight model, only to then have the U.S. government suddenly restrict access to that technology,” he says. “The initial security questions are difficult enough and require significant investment. Then, the bank is faced with a set of very difficult resilience planning questions. It’s critical to have those conversations and answer those questions, before adopting such a model.”
Previous