Why enterprise AI initiatives stall — and what CIOs can do about it

CIOs who launched AI projects over the past year that didn’t deliver on expectations, or failed entirely, aren’t alone. Investments in AI projects have surged, but nearly half of enterprises have struggled to demonstrate business value, according research from Gartner. “People jumped in with unrealistic expectations, and that’s where the biggest disappointments are coming from,” says Manav Misra, chief data and analytics officer at Regions Bank. He and many other practitioners say technology isn’t the problem, it’s everything that surrounds it. Organizational, operational, and structural failures have been derailing projects at every step in the journey.

While the CIO’s team may be clear-eyed on model selection, platform architecture, and data pipelines, in many cases the wrong projects were chosen, business outcomes and success metrics weren’t defined with enough granularity, the level of collaboration with business leaders fell short, or users weren’t brought on board from the start.

AI projects may also be plagued by user fear, uncertainty, and doubt. “We got paid to train our offshore replacements, and now we’re getting paid to train the AIs to replace us,” says Keith Fulton, CDO at banking technology provider Jack Henry. The reality, at least for now, is that while LLMs can improve productivity, they need users to review and validate outputs. AIs, he says, act like brilliant interns that need guidance.

“The playbooks of the past aren’t effective,” adds Afshean Talasaz, former chief technology and data officer at Colonial Pipeline. “You need to dig into the details of these things, figure out what you want to do, and make your expectations really clear.”

Don’t let any of these unforced errors become your next AI project’s Achilles’ heel.

Stop leading AI projects. Start co-leading them.

Sumeet Gupta, senior managing director, and AI and digital transformation practice lead at FTI Consulting, says that if you ask a CIO to run an AI CoE on their own, it’s almost never a recipe for success. “CIOs may often end up driving it from a platform-centric view, and that’s not where most of the issues lie,” he says as these aren’t technology projects, but rather business projects that happen to use AI to achieve desired business outcomes. “You need to be co-leading this with the right business owners in the enterprise you’re in, because that’s where the fundamental business problems get solved.”

Fulton agrees, saying AI projects need to be led by the business. “Business leaders find the opportunities first, and then AI experts can help them triage those possibilities to determine which should be on a shortlist,” he says, adding not to rely on consultants on this step. “You need an internal partner who will keep you conservative.”

Fulton learned this lesson early on. “The only failed experiment was a consulting engagement a year ago that turned into a fishing expedition using AI in the finance department, which failed to find any useful cases based on the technology at that time.”

Misra, on the other hand, has a team that includes product managers who come from the business side, so they have a clear understanding of business needs. They meet with business leaders and help them identify viable opportunities. “You need an executive sponsor willing to commit to the value and invest in building it properly, and the business owner must be clear about what success actually looks like,” he says.

The business owner should set the success metrics

CIOs should work with business leaders up front to determine and agree upon what metrics determine success in terms of addressing desired business outcomes. Both business and technology groups should review those metrics quarterly.

“The person who owns the P&L for that business should be responsible for the measurement of that and should determine what the metric should be,” Gupta says. “How it’s measured is something the CIO can opine on, but what that business metric is that’ll feed into the P&L should always be determined by the owner of the P&L.” But you need more than management buy-in to succeed.

Users need to be in on it or they won’t be on board with it

The best laid plans for any IT project will amount to nothing if employees won’t use the technology, and this is especially true for AI. Gupta recalls an initiative at a firm where they invested in building a wrapper around an AI model that didn’t end up delivering any unique business value. “The AI team in the CoE thought it would be interesting to develop, but didn’t bring the larger user base into those discussions,” he says.

There’s fear and cynicism about AI, Fulton adds. “The organizational change management aspects of getting people to see it as a helper and not a replacement is a big challenge,” he says. “But while AI can create recommendations, for example, human review is an essential verification step before delivering anything to clients.”

That’s why users need to be involved from the very start and at every step in the process. “I can’t tell you the number of cases I’ve been through where technology teams have built an AI product that did all these things, and the business users didn’t use it because they weren’t in on the process itself,” Gupta says. “That never works in a transformation, especially with technology.”

Misra recalls one business unit where they didn’t achieve the level of buy-in needed initially, which slowed down deployment. “Someone would say they’re not sure it’s going to work, and that would start a cycle of doubt,” he says. He recommends identifying and supporting early champions, creating workshops for their peers, and doing a phased rollout with quarterly measures of usage and adoption.

Engaging with users from the earliest stages has another benefit of the change not feeling as big because they see it happening in increments. And at Jack Henry, Fulton puts a heavy emphasis on coaching. “We have a team of 10 who help people use these tools,” he says. “If we guide them the right way we can win them over. But if it’s not a fit, we don’t coerce them.”

Ultimately, says Gupta, you need to convert users and make them accountable as part of the process. That means making them part of the discussion of how the AI might affect workflows and how any needed changes should be addressed.

Think through workflow and change management implications

Before using AI to automate, CIOs and stakeholders should review current and planned workflows and how they’ll impact productivity and how employees do their jobs. “Workflow redesign and change management go hand in hand,” says Gupta. “For example, there was an AI product at a firm that didn’t get used for a long time because they didn’t do any change management around it.”

The company designed a fairly complex agentic AI workflow to eliminate a high degree of manual labor and improve process accuracy in a crucial part of their business operations. But for the workflow to be properly adopted, they had to address some workforce-related operating model and change management issues. “This wasn’t done in a timely manner,” he adds. “So, while the AI workflow itself was well conceived and properly developed, it wasn’t effectively used for many months.”

He applies what he calls first principles to each project: know the problem you’re trying to solve with the workflow, the desired outcomes, and what your inputs are, then rethink how it might work in the future with AI. “You have to design it based on those constraints,” he says. “A lack of reinvention or reimagination around AI is issue number one.” That requires engaging with both business leaders and users.

Fulton at Jack Henry has a different view to not redesign everything at first, but start with task automation and build from there. “Business process reengineering has been around for 20 years but has never lived up to its potential,” he says, because it’s expensive and large companies have hundreds, if not thousands, of business processes. Today, he adds, agentic AI works better on simpler business processes than on more complicated ones.

That said, agents are starting to understand business processes and optimize them. “Instead of building a workflow diagram with an RPA tool, you just tell the LLM what the steps should be and it determines its own workflow for that,” Fulton says. However, agents capable of autonomously optimizing complex, multi-step enterprise workflows without extensive human configuration are still in the early stages and are unproven at scale.

The best intentioned workflow designs and organizational change management plans may not work, however, without the right understanding of the data requirements.

Data readiness anxiety paralyzes projects

Conventional wisdom about having all of your data ready before embarking on an AI can stymie projects before they get started. “Many companies think if they don’t have the data, it won’t work,” says Gupta. But not all projects require pristine data. Too often, companies stumble on data concerns that don’t apply to their actual use cases.

“Data is a big issue for machine learning models but not for most large language models,” Fulton says, unless the project focuses on complex forecasting, customer-level personalization, or sales lead scoring, for instance. But if you need the data in those cases, Misra says to identify the specific data the project requires, and focus only on that so the data problems you have become easier to solve due to AI-based data cleansing, integration, and preparation tools.

There are, however, plenty of legitimate data concerns that should give pause, says Avivah Litan, distinguished VP analyst at Gartner. These include data quality, metadata, and lineage gaps that prevent compliance, explainability, and regulator reporting, as well as siloed datasets and immature metadata management that block regulatory readiness.

AI can be smart, but not always reliable

Talasaz says governance is a big deal around orchestration and observability. “Agents can do things they weren’t intended to do, and you need visibility into that,” he says.

The root cause of big failures is companies ask too much of LLMs and trust them without verification steps, adds Fulton. LLMs, he says, bump into things, make dumb mistakes, and lack context about specific domains. “When assessing outputs, don’t confuse smartness with experience and context, so every AI output needs human review before production use,” he says.

For more complex tasks, an LLM can also lose its way when attacking problems that require more context than the available token window can handle. At that point, everything gets compressed to fit, and the LLM can lose its rhythm and start hallucinating. “LLMs can do a large number of these things, but the bigger they get, the more they lose track,” Fulton says.” He’s found that, in practice, shorter, more bounded tasks produce more reliable results than long-running autonomous processes, and he’s structured Jack Henry’s AI use accordingly.

Failures may also come from unsanctioned projects that come out of left field. “With shadow AI, the concern is someone builds something for critical work processes in an ungoverned way,” Talasaz says. “Then when it’s wrong or it breaks, there’s no observability, creating sustainability challenges and risks for the organization.”

While a Gartner survey says 69% of organizations suspect or have evidence of employees using prohibited AI tools, Jack Henry has taken steps to minimize that risk by maintaining strict data governance through mandatory training and approval processes. Policies strictly prohibit unapproved tools, such as public chatbots, and the company provides more than 100 AI-based applications so users hopefully don’t feel the need to go elsewhere.

Too many PoCs make an untimely demise

Pilots can fail when there’s a disconnect between those who run the pilots and those who expected something different to come from it, Talasaz says. “Be clear about what the pilot is intended to do, whether it’s more about experimentation or if there’s a high degree of certainty of getting an intended outcome,” he says.

Some pilots move forward without a complete understanding of the business benefits. Gupta recalls one firm that embarked on an AI initiative without conducting a proper economic analysis on the financial value of the project. This, he says, is an important go or no-go stage that should be undertaken after an AI use case is conceived, but before the business deploys capital into it. “When we came in and did the financial analyses, we found that the AI initiative’s one-time and ongoing operating costs would be more than the median projected savings,” he says.

Other projects fall short of what’s needed. The pilot might work 90% of the time, and getting to 99% might take six months of tuning and cleaning up the data. But financial services businesses like Jack Henry require 100% correctness, Fulton says. “A business process automation tool that works 99% of the time is not valuable,” he says, since just one mistake can lose the confidence of business leaders, leaving a pilot dead in the water.

It’s not uncommon to discover that how engineers built a PoC doesn’t work at scale or is just too expensive at scale. “You can build a PoC but not have the right design and engineering talent or infrastructure to scale it,” Talasaz says.

Another potential setback is time. The original team often goes back to their day jobs once the pilot concludes, taking their knowledge with them as a new team takes over, and that can slow down the project. “I like to have the same team take it from PoC to production,” Talasaz adds. Gupta recommends having explicit proof-of-life milestones, user simulations of agentic workflows, and accuracy thresholds that the PoC must achieve before it can move from one stage to the next. “If you don’t have the right people involved and the right milestones, that’s why these pilots get parked,” he says.

Production attained but operational sustainability is unsure

Your pilot is now in production and it works — for now. So how do you maintain momentum? “Most teams haven’t fully considered sustainability if they’re having trouble getting projects into production, but you have to think about that,” Talasaz says.

Monitoring for ongoing operational issues such as model drift are important, but so are more fundamental issues like risk of vendor lock-in and rising technical debt that can delay and add expenses to upgrades down the road. If unaccounted for during the design and pilot phase, these issues may be baked in by the time you’re in production.

Gartner warns against allowing your data, models, or workflows to be locked into a single vendor’s APIs, data lakes or platform tools. Instead, follow open standards, and use open APIs and modular architectures in AI stack design. As for technical debt, Gartner predicts that in the next four years, 50% of enterprises will face delayed AI upgrades as well as rising maintenance costs due to unmanaged gen AI tech debt. It recommends enterprises maintain an AI registry,  enforce model cards, implement drift monitoring, and require vendors to provide model-change notices. In other words, apply the same IT lifecycle management disciplines you would to any other IT project.

Devil in the details

There’s a lot of figuring out to do with AI projects, but the key to success, Talasaz says, is in the detail work, not only on the technology side with the data, governance, and so on, but on how things work in the business. “Get into the nitty-gritty of what the work entails,” he says. “Work down from the desired business outcome and up from the technology foundation.”

And if the technology’s part of the problem, don’t discard the idea. The timing just might be off. “Last year we tried several different code assist tools for developers, and they weren’t capable of working on systems as large as ours,” Fulton says. But eight months later, the team tried the same tools again, and now adoption is like wildfire. “Sometimes it’s not never, it’s just not yet,” he says.