At Fujifilm Biotechnologies, an AI agent designed to help the legal department read contracts is gearing up to graduate from sandbox to full production in early December.
Contracts are a large part of the business at the CDMO, which partners with pharmaceutical and biotech companies to develop vaccines and therapies. The company’s legal team devised an agent to review contracts, check standards around terms and conditions, and propose edits for the legal team’s review.
“That’s a perfect use for a set of agents,” says Fujifilm Biotechnologies CDIO Mike Tomasco.
Similar types of AI agents that help enterprises and employees tackle time-consuming or repetitive work are cropping up in vast numbers across organizations. Gartner anticipates that by 2028, Fortune 500 companies will have on average more than 150,000 agents. Microsoft alone has deployed more than 500,000 internal agents, and HR, payroll, and workforce management platform UKG now has more agents than employees, according to its chief transformation officer, Prakash Kota.
As enterprise tools have made it easier for employees to spin up agents, CIOs are tasked with managing the lifecycles of agentic tools, advancing them from development to production, continuously measuring them against KPIs, and sunsetting them if they’re no longer meeting an organization’s needs.
As a result, IT leaders are thinking and talking differently about agents, says Madhu Narasimhan, CIO of healthcare organization DaVita.
“The conversation among CIOs has shifted from how do we build agents to how do we make sure they continue delivering value,” she says. “Getting an agent into production is no longer the hard part.”
Bringing agents to life
Managing the lifecycle of an AI agent starts before the build. CIOs and their teams outline the foundation and guardrails for employees to build, which then allows CIOs to govern agents through their lifespan and, if needed, eventually retire them.
Kevin McDonald, principal consultant at tech consulting firm Hylaine, advises leaders to define the job of an agent and build it with narrow permissions. “Treat an agent like a product with an owner, not a project with an end date,” he says.
Kota adds it’s important to balance IT visibility with employee autonomy, and recommends a playground where employees can experiment. He also recommends tech leaders assign dedicated identities to agents that are separate from user credentials and with a narrower scope of access.
At Fujifilm Biotechnologies, Tomasco wanted to take a deliberate approach to agentic AI, given the company’s position in a highly regulated industry. He didn’t want unfettered access to Claude Code like some companies have allowed.
That led him and his team to build an isolated sandbox called Genki Studio to encourage employees to experiment with agents without building them in the Wild West.
It’s a bring your own data environment with enterprise security controls, restricted network access, role-based permissions, and auditing, Tomasco says. All of the company’s 5,000 employees are eligible to apply to be part of the studio, though only a small portion have an interest and skillset in developing AI solutions. Fujifilm Biotechnologies prioritizes applicants with a relevant use case, strong interest in AI, and experience or familiarity with data and analytics.
About 50 employees currently work in the studio, where they build prototypes of agents using leading AI models and the same tools developers have. Following the build, an employee invites coworkers to test the agents. Then, there’s typically a domain-based review to ensure the agent follows regulations and doesn’t duplicate something that already exists or is on the roadmap.
If all those factors line up, the agent can be released into production. The timeline from idea to release is typically six to eight weeks, according to Tomasco.
Measuring agent success and value
Once an agent is up and running, Fujifilm Biotechnologies assesses its utility over the next several weeks to determine if the agent should stay in production. It uses metrics such as adoption and KPIs specifically related to manufacturing processes, such as faster time to market for a pharmaceutical treatment.
The organization also checks on agents throughout their lifecycle to ensure they’re still fulfilling the originally designated purpose. “Agents can drift from the guardrails that you set up for them,” Tomasco says.
Meanwhile at DaVita, each agent goes through an internal AI intake and evaluation standard, Narasimhan says. The company spent over a year building a security and governance foundation before launching a company-wide AI platform this spring. To date, employees have created more than 10,000 agents, including ones that navigate questions on benefits or regulations, says Narasimhan.
“We assess compliance, cybersecurity, governance and business value before broader adoption,” she says. A team that includes staff from clinical, legal, cybersecurity and more will review each AI use case. KPIs for successful agents include accuracy, reliability, adoption, and business impact. If the agent is redundant or lacks engagement, DaVita considers decommissioning it. “I don’t believe in technology for technology’s sake,” she says.
Hylaine’s McDonald adds CIOs can assess agents on time savings, cost per task, and how frequently a human overrides an agent’s output. When override rates start to climb, or if running the agent costs more than the work it’s replacing, it may be time to sunset the agent, he says.
He recommends organizations maintain a registry of every agent with its permissions, versioned prompts, and tools. That way, when it’s time for agents to retire, CIOs can fully revoke access and prior context.
“Forgotten agents with live credentials are a real risk,” McDonald says.
Tomasco agrees that you have to be willing to just shut it down if an agent doesn’t hit established metrics. “There’s no sense in having something run that’s not creating value.”
The business, front and center
The topic of value, and whether AI agents deliver it, is increasingly top of mind for CIOs. Instead of measuring success by adoption or how many employees are building or using an AI agent, Kota says the focus now is how to convert that into business outcomes.
Employees spending their days experimenting in sandboxes or trialing agents doesn’t automatically translate to increased productivity or results for the organization as a whole, he says. “If we don’t impact the top line or reduce the bottom line impacts, what’s the purpose?”
For organizations like DaVita and Fujifilm Biotechnologies operating in high-stakes industries, use of AI agents speaks to a broader purpose to boost efficiencies that can save time for biopharma customers and patients.
“It’s exciting to be a part of it when you can help bring medicine to market faster for people,” Tomasco says. “It can be life-changing.”