Financial institutions that are accustomed to paying fixed-term subscription fees for large-language-model (LLM) artificial intelligence services should start preparing for a change that will make their AI budgeting and forecasting much more complicated moving forward. A new pricing model that isn’t based on a set time period or specific number of users, but on a much more variable new unit of measurement — the AI token — is already on the rise.
Lamont Black, the founder of Wide Open Ventures, who provides AI consulting services to small financial institutions, describes an AI token as a unit of information. “So, if you think about large language models being able to process these large documents, websites, all the different information that gets thrown at them in order to do the analytics around that, they have to break it into these units, and sometimes that’s a word or a group of words,” he says.
The LLMs break down both the information being put into them from user prompts, as well as the information they send back out, into those units. Each unit represents a token. That means the more words you put into an LLM and the more text, images, code and decision-making it consumes in order to respond, the more tokens you use. The large LLM providers such as OpenAI (ChatGPT), Anthropic (Claude) and Alphabet’s Google (Gemini) figure out their own costs per token, with rates often based on the LLM and the service it’s providing.
Nicholas Merizzi, AI infrastructure leader for Deloitte Consulting, who co-authored an April 2026 report on AI tokenomics, says it isn’t hard to use a lot of tokens very quickly. “At this stage, even very basic prompts and simple tasks can easily add up to 5,000 to 10,000 tokens,” he says. “And as we look to have more autonomous [AI] agents do more complex tasks and more thinking, that type of reasoning that we’re asking the models to do can easily go to 40,000 to 100,000-plus tokens with every ask.”
Before 2026, those LLM providers priced very few services based on how many AI tokens a customer actually used. Microsoft Copilot, the generative AI tool of choice for many banks and credit unions, incorporates ChatGPT as its LLM, and has also shied away from token-based pricing.
Filippo Scognamiglio, a managing partner at Boston Consulting Group (BCG), who advises technology leaders across financial institutions and recently co-authored a series of BCG reports on the true costs of AI tokens, says those LLM providers have essentially been charging customers a discounted rate up until now. They did so in order to grow their market share and increase their companies’ valuations, but the bill is now coming due.
“There’s going to be a point where bondholders are going to ask for a principal back and equity holders are going to ask for their quarterly earnings, and somebody’s going to have to fund that,” Scognamiglio says.
Sonata Bank, a $283 million institution based in Brentwood, Tennessee, which specializes in banking franchise restaurants and providing sponsor banking services to fintechs, uses multiple LLM models and agentic AI tools. Will Rhoads, the bank’s chief information officer (CIO), says he’s well aware of the change that’s coming. “We know that the $200 we paid to Anthropic for Claude really cost them several thousand dollars to provide,” he says. “That’s not going to last. Same with ChatGPT, same with Copilot — it’s not going to last.”
Merizzi says the change is starting to impact the bottom line for some businesses. “I have seen enough of these client bills that go from November-December of last year to May-June of this year where you’ve had 200% increases month over month,” he says. “And so, this has been what’s kept a lot of CFOs up at night.”
Agentic AI Costs More
While most simple generative AI searches and tasks like summarizing documents or drafting emails are still covered under those LLM subscription-based pricing models for now, token-based pricing is already in effect for more advanced services, such as those powered by agentic AI. Microsoft announced a usage-based pricing model effective July 1 for Copilot Cowork, an agentic AI assistant that started as a free service in March for businesses with a Microsoft Copilot 365 license.
Black says it simply takes more horsepower to run AI agents. “To the extent that it’s decisioning, taking actions, these are things that are going to require much more high processing and [put] higher demand on those models,” he says.
Rhoads says Sonata Bank’s subscription-based model of Claude puts limits on the number of times a user can utilize its agentic AI tools, requiring them to pay for each token used after a certain point.
Merizzi says he is seeing that scenario play out more and more. “We’ve been witnessing this hybrid, where you pay some base and get some amount of allocation on either a per user or enterprise-wide basis,” he says. “But just like your utility bill at home, the more you consume, depending on how much water you run, is how you’re going to pay.”
Scognamiglio says such pay-as-you-go models will make it harder for financial institutions to forecast their AI usage costs moving forward. “The price for both input and output tokens varies widely on a model-by-model basis,” he says. “So, there’s no way to directly compare and say, ‘OK, let me compare Claude Sonnet 4.6 with Gemini 2.5 Flash and see how the tokens compare.’”
Strategic Planning Is Key
Banks and credit unions can take steps to get ahead of this issue. Merizzi and Black say they need to start with their 2027 budget planning, and develop strong coordination between the CIO and CFO.
“It’s the right combination,” Merizzi says. “A lot of this is hygiene on how you design your prompts, what models you allow, who [is allowed] to use them, what models are enabled by default versus consciously forcing somebody to go and pick something else that may be more expensive. And so, there’s joint accountability on tackling the problem.”
Black says CIOs and CFOs should also be talking with their institution’s AI vendors about any anticipated changes in models or pricing. “There should be a clear pricing structure of this many tokens, this many dollars and here’s the run rate,” he says. “So, if you’re going to do that financial forecast, then you have to have some idea of that.”
Rhoads has set up a system at Sonata Bank that ensures employees are always routed to the lowest-cost AI model available that can still help them complete their task. “Being able to assign that by employee, by department, is really important for us because then we can set budgets,” he says, adding that the bank also blocks employees from using more expensive models for certain tasks. “We don’t want employees burning our tokens to plan their vacation.”
Black says a multifaceted approach that protects the bottom line while encouraging innovation is the sweet spot where financial institutions should try to land. “You want to figure out, ‘Is this an effective use of that person’s time and our resources?’ And if so, give them more money, raise the caps,” he says. “But if not, [ask] ‘How do we train them and do some redirection?’”
Previous