For banks navigating rapid digital transformation, pilot programs can serve as both a safety net and a springboard. They allow institutions to test new technologies in a controlled environment before committing significant time and capital. Done right, pilots help mitigate risk, align stakeholders, and accelerate decision making.
Dilip Venkatachari, senior executive vice president and chief information and technology officer at U.S. Bancorp, says pilots, proof of concepts, and “fail fasts” are essential when testing new technology in banking.
“The approach and construction of them may look different because of various evolutions in the last decade but overall, I don’t see them being less utilized,” says Venkatachari of the Minneapolis, Minnesota-based bank. “At U.S. Bank, we use a mix of approaches, tailored to each situation designed to minimize risk while maximizing pace and business value. Technology, digital, and product transformations have enabled us to experiment quickly and help accelerate decision-making.”
Furthermore, not doing a pilot that involves a limited set of real customers and only relying on a proof of concept or employee testing could lead to a “false sense of confidence” when it comes to implementing new technology, notes Nicole Lorch, chief operating officer at $6 billion First Internet Bancorp in Fishers, Indiana.
“Not every project goes perfectly and sometimes there are unintended consequences,” she adds. “If you are only using your own employees [to test potential new technology], it may not be enough. You need a wide range of different people and demographics. A pilot should always include a small subset of your actual user base.”
Pilots are also essential tools in rooting out a good product from a mediocre one, akin to taking a new car for a test drive, says Paul Schaus, managing partner and founder of research and consulting firm CCG Catalyst.
“A pilot is like going car shopping,” he elaborates. “If you are only buying the car based on what the salesman is telling you, and not testing it for yourself, there’s a good chance you’ll run into issues down the line.”
Sometimes when banks make decisions to implement new technology, even if they don’t necessarily regret it, the new technology may not live up to expectations.
“That’s why you need pilot programs,” says Schaus. “The great thing is they are easy to get up and running pretty quickly. You can get real customers and real employees playing in that system and seeing how it works.”
In Schaus’ opinion, pilot programs are happening less across the industry than in decades past. He points to several factors contributing to this such as the established bank technology vendors consolidating their market power and adding exclusivity clauses to contractual language. Newer and startup vendors trying to compete against established players are more willing to engage in pilot programs, Schaus says.
“To me, it seems like banks are not doing pilots as much as they used to,” he adds.
Lorch says First Internet Bank is “in some cases actually doing more pilot programs than in the past” but adds that if there is a decrease in pilot programs, it is because of vendor consolidation and many products “already being very well-known commodities.”
Pilot Program Best Practices
Setting clear goals, KPIs and knowing exactly what you want to measure are keys to running a successful pilot program.
Banks also need project champions that can build internal support for the project.
“Building internal support starts with engaging all impacted stakeholders from the outset,” says Venkatachari. “Make sure they understand the business case for the change as well as the intended outcomes. Time, scope and cost bound your pilot program and limit ‘what if’ scenario planning. Then, evaluate your success criteria through customer-centric principles.”
Lorch emphasized the importance of discipline and communication when running a pilot.
“It starts with setting clear, measurable objectives and making sure the pilot team understands the KPIs,” she said. “You need internal champions and a well-defined scope, otherwise, every ‘wouldn’t it be nice if’ adds time and complexity that can delay rollout. We’ve seen vendors overpromise and underdeliver, and sometimes you only discover your non-negotiables when you trip over them. That’s why it’s critical to know when a pilot has gone on too long. It’s okay to fail, but you want to fail fast, conduct a postmortem, and use those lessons to shape communications and expectations for what comes next.”
A successful pilot also comes down to the human factor, notes Lorch.
“You know who your key contacts are at the vendor, you have developed relationships with these folks,” she says. “Be clear with them about the expectations and the outcome you are looking for. That way everyone understands the same set of expectations and is working towards the same goal. It’s important to be open, collaborative and ask a lot of questions.”
Venkatachari agrees with that sentiment.
“Make sure you have clear terms around how and where data will be used and shared, and ensure appropriate controls are in place to mitigate a potential data loss or a breach,” he adds. “Outcome based agreements that offer incremental delivery and account for economies of scale are also beneficial.”
Lastly, he says, focus on creating a long-term vendor partnership that extends beyond the pilot program, and includes collaborative ongoing support and strategic guidance to handle future challenges that may arise.
If implementing a major new tech product, such as a new consumer digital platform, banks can also roll the new product out in waves, such as starting in just one geographic area, to ensure any bugs are worked out before doing a full rollout, says Clio Silman, a managing director at Cornerstone Advisors.
“We are very pro banks doing pilots, especially if they are doing something risky like implementing a new core system,” she adds.
Risk Factors
Indeed, full-scale technology replacements can be risky without first running a pilot.
“The key risks you need to evaluate or be aware of are how disruptive this change is to critical systems, what could go wrong during customer migration, and if full implementation is truly cost efficient,” says Venkatachari. “A pilot helps you identify potential issues on a small scale or controlled environment, which minimizes negative impact.
Pursuing incremental modernization also provides the ability for banks to build or modernize their technology as needed instead of spending dollars that don’t produce a full return, Venkatachari adds.
“It’s also possible to underestimate the process involved for full-scale replacement and the impact it has on your employees and the culture,” he notes. “Successful change management is key. Ultimately, the decision on whether to start with a pilot or proof of concept depends on a variety of factors, including how broad of an impact the change has, the level of skill change needed to support the new technology, and the criticality of the system or service.”
Without a pilot, banks have to start doing testing the day a new product goes live, which can lead to negative outcomes, says Schaus.
“If you have an undertested tool and then it has some problems once you go live; it’s hard to roll it back once it goes live,” he adds. “Then you’ve got clients that can’t access their funds, or some functionality is down and that’s how you lose clients.”
Previous