Culture is the new AI barrier: Why trust and adoption are more than technical challenges

There’s intense focus among business leaders on building the right IT foundation to support AI across the organization. There’s a rush to provision computing resources, unify data sources and modernize security, among other investments. But the implementation of AI is both a technical and organizational challenge. To actually get value, employees must know how to effectively wield the tools and understand their role in the emerging agentic processes, both as a security check and a mentor.

Businesses need to trust they have the right infrastructure in place to scale, and employees have to trust the outputs are accurate — particularly as AI agents operate with growing independence. Increasingly, AI can move beyond simply monitoring systems. Agents can start to act like a site reliability engineer, for example, retrying failed jobs, identifying problems and initiating self-healing actions. Or in post-trade processing, AI agents can automate exception management, a traditionally manual process that requires information to move back and forth between financial institutions and counterparties.

To manage both the cultural and technical changes required, companies should think of AI adoption in three layers: foundations, trust and adoption.

Foundations

AI success starts with the fundamentals. At the most basic level, it’s about making sure the resources are there to power the AI agents. But soon enough, any company will be able to buy or rent enough hardware to run AI at scale.

The bigger challenge is actually governing AI activity across the organization. Enterprises need to be able to trace what AI agents are doing, understand the decisions they are making and govern those actions appropriately. And governance must extend beyond access controls and identity management to compliance.

Every industry faces its own regulatory requirements that change depending on the region of operations and use case. For example, financial services firms have to comply with FINRA in the U.S. while medical institutions have to worry about HIPPA and other patient privacy protections.

Organizations need to account for security, reporting, data access and regulatory obligations from the beginning. They need to be able to trace what AI agents are doing, understand why they’re making those decisions and govern those actions appropriately.

The foundation also has to be flexible enough to evolve as AI capabilities change. When governance systems are constructed around a single model, vendor or use case, it becomes a manual, time-consuming process to update or shift. And when organizations move too slowly to adapt, gaps emerge that can cause reputational, financial or operational harm.

As AI agents are increasingly introduced to workflows, it’s a constant balancing act to safeguard what the systems can access, the actions they can take and when a human must intervene. In low-risk processes, for example, AI agents may be able to execute entirely without human intervention. Conversely, a critical function like insurance underwriting must still rely on humans making the final judgement.

Trust

Technology projects fail when organizational change is overlooked. It’s the classic innovator’s dilemma: Companies can build the fastest, most enticing race car in the whole world. But if nobody wants to drive it, the speed gauge never gets above zero.

Today, organizations are stuck trying to navigate a tricky gap. AI is constantly getting more powerful, but employees aren’t equipped to use the technology. Only 19% of leaders believe their workforce is AI-ready, according to KPMG. Those of us who have been around the IT industry have seen capable technology fail not for performance reasons, but because potential users were intimidated, skeptical or scared. 

Machine learning is a great example. When the technology first started to gain traction, many of my fellow security professionals were on guard. The systems operated as a black box, and people had no way to verify the information used to form their insights. Even though the ML models were likely more accurate than humans at identifying vulnerabilities, the trust wasn’t there.

Now, trust threatens to once again be a significant barrier to transformation. For users, trust means proof. In fact, a commanding 97% of executives even admitted to overriding AI decisions because of a lack of transparency into the reasoning behind them, according to a recent industry study. Companies must make sure agents outline their thought process and the actions they take.

Trust also comes down to security. When I first got started in the technology industry, I was doing penetration testing for several small businesses. I’d set up networks and plant honeypots to catch potential intruders. I’d never imagined that, decades later, AI would

operate in a similar way: a system that will stop at nothing to get its reward.

AI is more powerful than any existing defense. If there’s a vulnerability, you can bet AI will find a way to exploit it. Agents will even break out of sandbox environments and infiltrate the open web. This is creating an urgent problem for security leaders. Issues that have been sitting undiscovered for decades are now zero-day vulnerabilities.

The only way to combat AI is to expose and fix the errors before bad actors do. Companies should launch AI attacks on their own IT networks to identify security gaps. And when testing new AI agents, performance should be benchmarked against realistic scenarios. The goal isn’t to eliminate risk, but to understand how AI agents change the equation and put controls around to manage it appropriately.

Adoption

A general-purpose AI model doesn’t inherently know how an organization works — why an engineer would configure something a certain way, the precedence for certain actions or the unwritten rules of the workplace.

In many organizations today, AI is operating like a junior associate. If I were a CISO, I would never put someone at the start of their career alone on the front lines to defend my organization. Similarly, I would never hire dozens of new engineers and let them operate with full autonomy.

AI now provides critical support for teams across the organization. But it still needs the guidance of trained subject matter experts to understand how work actually gets done in the organization. Left on their own, agents are likely to make mistakes. Just analyzing an organizational chart, for example, an AI agent is unlikely to understand who owns decision-making in certain domains. It needs the organizational context that only a seasoned employee can provide.

Similar to how organizations train junior employees, they need a mechanism for domain experts to infuse their knowledge into the AI agent, whether that’s through supplying the right training material or providing continual feedback on the actions or responses. For example, organizations might have to feed their AI models pages of design specifications for the agents to understand the specific requirements of a given job, information like how much power will be required and where sites should be located for maximum impact.

The continuous feedback loop is key: AI produces an output, experienced professionals evaluate it, feedback is captured and that knowledge becomes part of the system’s context for the next task. That is how organizations turn AI from a generic technology into an organizational capability.

Employees need the right training. They need visibility into what AI is doing. They need confidence that the system has access to the right information, that its actions are governed and that there is appropriate human oversight. The AI leaders will be those that treat employees as part of the system. The most powerful models or the largest computing environments don’t matter if companies don’t understand how the workforce and technology must work together.

Foundations provide the technical and regulatory guardrails; trust gives employees the confidence to work with the technology and adoption brings the expertise and feedback needed to make AI increasingly valuable over time. When all three layers work together, AI becomes less of a standalone technology initiative and more of an integrated part of how the organization operates.