At the TechEd developer conference in Berlin, SAP presented additional details of its “Autonomous Enterprise” concept, which includes AI capabilities via Joule Work and SAP Autonomous Suite, built on the SAP Business AI Platform. The company’s focus is not just AI, but AI agents capable of acting automatically within the SAP system by performing analyses and answering queries.
This raises questions, such as: How can the costs of AI agents be controlled? Who is allowed to develop the agents, and how can companies maintain oversight? Jonathan von Rüden, chief AI officer of SAP, explains in an interview with Computerwoche how the software provider is addressing these questions — and where work still needs to be done.
Computerwoche: Companies are increasingly using AI and developing more and more AI agents. But it is often difficult to predict how costs in this area will develop over time. How is SAP addressing this?
Jonathan von Rüden: We generally proceed in three steps. First, we clarify together with the customer or in product management which use cases are relevant and what should be achieved with them.
We distinguish between different areas of focus. First, there’s general productivity: transcriptions, summaries, presentations, or information research. This now extends far beyond classic office tasks. For example, information from the internet can be combined with SAP data and used for financial analysis. However, this area is difficult to define precisely because it encompasses so many tasks in everyday work.
In addition, there are dedicated AI agents that are centrally deployed and managed by IT. These can be standard agents from SAP or customer-specific agents from Joule Studio.
Finally, there are use cases where companies can differentiate themselves significantly. Here, they should consider whether they can develop their own solution.
Based on this information, we estimate consumption. To do this, we consider factors such as the number of employees, how frequently a process is carried out, and how many documents are processed. We also incorporate empirical data from comparable customers and industries.
Ultimately, a large part depends on usage. However, budgets and limits can be set to control costs.
How do these limits prevent costs from spiraling out of control?
In Joule Work, for example, a daily usage limit can be set for each employee. The user can see how much of this limit has already been used via a percentage indicator. Companies can assign different budgets, for example, for individual departments or countries.
If the budget is exhausted, employees can request an additional allocation if, for example, a task absolutely needs to be completed. This allows usage-based billing to be combined with centralized cost control.
The subsequent success measurement is an optional step. Agent and process analysis can be used to verify whether the desired effects have actually materialized.
Many users, as a precaution, opt for the most powerful model — even if it’s the most expensive. How can this be controlled?
The default setting alone has a significant impact. We see this in our own use within SAP: Most people don’t change the default model. If I set a default model that balances costs and performance well, this can already reduce costs considerably.
Companies can therefore consider: Do these employees really always need the most powerful model for their tasks, or is a cheaper one sufficient for everyday use?
Budgets also create an incentive to engage with the issue. If I know that my budget is limited, I’m more likely to consider whether I really need the most expensive model for a particular task.
Couldn’t this selection be made automatically?
We’re working on that with model routers. However, auto-routing is more complex in our use cases than in scenarios where, for example, it’s just about summarizing documents. When a system has to search through and link various transactional business objects, the question of the appropriate model becomes significantly more challenging.
We are currently testing automatic routing internally and plan to make it available to our first customers in the future.
SAP supports 70 LLMs in Joule Work. How much influence do customers generally have on the model selection?
That depends on what they’re building or using. Anyone developing their own agent can choose the model from the available options. In Joule Work, the selection is made via modes like Standard, Extended, or Light; in addition, there’s automatic routing, which we’re currently working on.
We can replace the LLMs behind these modes if a different model is better suited for the respective task. The user then doesn’t have to constantly deal with individual model versions.
For our assistants, the developers decide which models best perform the tasks. They conduct extensive evaluations to determine this. When a new model is released, they check whether it actually improves the assistant. Only then is a replacement worthwhile.
Additionally, we take geographical requirements into account, for example, if customers want to use exclusively European models. In doing so, we must consider regional regulations while simultaneously ensuring that the models offered are suitable for the respective tasks.
What about local models?
For a purely web-based agent, a local model generally plays no role. However, in the desktop application, the models for natural language processing, for example, already run locally. We are working on supporting additional local models.
This is also interesting for users because it allows them to save tokens. For example, they could say: “My token budget is used up, so I’ll continue working with local models.” However, the hardware must be suitable for this. Furthermore, IT departments are often more restrictive when it comes to what can be installed.
On development machines, this is usually not a problem. However, rolling out such solutions on a large scale at a customer’s site is significantly more challenging. We are working on that as well.
Joule Studio makes it relatively easy for non-developers to build agents. How do companies prevent uncontrolled agent proliferation?
First of all, not every user automatically has permission to use Joule Studio. This role is assigned by IT. That’s the first level of control.
At SAP, we’re also initially rolling out Joule Studio to only a few thousand employees outside of IT and development, but not to everyone. We’re starting with those who are already familiar with it and have completed the necessary training. Each company decides on the specific access rules for itself.
Furthermore, not every personal customization needs to result in a company-wide agent. Individual skills can be a first step. This allows you to define, for example, the style in which an agent should deliver results. In this way, I can adapt it to my own workflow.
If something emerges that is also useful for others, I can discuss the next steps with IT: Will I be given permission to develop and share an agent myself? Or should I hand the idea over to a Center of Excellence that will further develop it?
Is there a specific approval process?
In Joule Studio, I can initially share an agent with specific colleagues who can test and work with it. Further access can be controlled via roles and the identity provider. Version control and the ability to later distribute agents to larger user groups are also included.
In the SAP AI Agent Hub, I can see the inventory and the respective status: Which agents need to be reviewed, which have already been reviewed, and which have been rejected.
The lifecycle can also include company-specific approvals. For example, SAP requires approval from the works council before a central agent is rolled out. Similarly, it’s possible to plan when an agent will be decommissioned. The associated IT components are also visible in this process.
How do companies determine whether an agent works reliably and provides value in day-to-day operations?
We combine development, monitoring, and process analysis. This allows us, for example, to analyze the usage and performance of an agent. Connecting to SAP Cloud Application Lifecycle Management enables deeper insights into operational data.
Agent mining and process mining then allow us to examine which decisions the agent makes, which process paths it follows and how often, and how much time each step takes. I can then compare these results with the existing process.
Testing is also part of the process. Tests are automatically generated from the specifications and requirements, which developers can then adapt. The results from testing and operation are fed back into further development.
This creates a cycle: I develop a customization or an agent, share it with others, test its performance, and improve it. Connecting all these components is complex. But for customers, this has a crucial advantage: They don’t have to assemble the individual building blocks from multiple providers.
To ask a somewhat heretical question: Where do you still see gaps?
We’ve made a good start with Financial Operations. Companies can see how many tokens, AI Units, or Agent Actions an agent consumes. We can still improve by directly linking this consumption to the value created.
Because a CFO rightly asks: We’ve spent this amount; what did it bring us? Agent mining can be used to investigate this, but today you need to delve into the analysis in a targeted way.
A more dynamic link between budget and usage would be interesting. For example, if a department currently needs more virtual support, budget could be reallocated there. Budgeting, value comparison, and the connection to the profit and loss statement are areas where there is still considerable potential for development.
Are customers already demanding this transparency?
Yes, we’re getting feedback that we can do more here. However, customers are at different stages. Some don’t yet have the problem of overspending. Others are already at that point.
We ourselves are also investing significantly, which allows us, for example, to work much faster in development. Nevertheless, we need to become even more precise in managing this investment.
Can SAP agents and agents from other providers be managed in an equivalent way?
That depends on what information and interfaces the respective provider offers. There is still no uniform standard for governance and policy enforcement. Currently, this has to be implemented manually. To close this gap, we have announced the SAP Agent Gateway. It simplifies controlled and governance-compliant communication between Joule agents and tools and is available free of charge.
For enterprise-wide runtime control and transparency, there is the SAP Enterprise AI Gateway. It extends the governance functions of the SAP Integration Suite and SAP AI Core and enables the enforcement of policies across all agent interactions.
However, if I want to define more complex rules, I need additional functions. For example, if a transaction exceeds a certain amount, one agent might only be allowed to perform specific actions, while another might not be allowed to take any action at all. The premium functionality of SAP Enterprise AI Gateway is available for defining such policies.
How many agents will companies use in the future? Providers and users alike are currently outdoing each other with the number of AI agents.
You could also call that “agent maxing.” It strongly depends on the definition. For example, if I develop something and Claude Code starts 17 subagents that are active for an hour and then shut down: Do they all count? Then we would probably already be at billions just from our own development.
However, the situation is different when we talk about reusable agents that are permanently available. A rough rule of thumb could be: Let’s take a company that uses roughly a hundred different SaaS applications, whose tasks are initially covered by 10 to 30 agents.
Over time, these agents will become more consolidated. Then, there won’t necessarily be a one-to-one mapping; instead, one agent will take on tasks across multiple applications.
The fundamental business processes remain the same. What changes is how we perform these individual tasks. In the future, they will be handled more extensively by agents or by a combination of agents and other automation solutions.
Are agents always necessary for this? Often, classic automation should suffice.
Absolutely. That’s why Joule Studio also offers the option of building a completely normal, classic workflow with n8n. If a task follows clear rules, a classic workflow is likely to be cheaper and faster.
However, hybrid forms are also possible: For example, I might have 17 workflow steps, but only need additional intelligence in one of them. I would only use an agent for that step. This creates an agent-based workflow that selectively combines classic automation with AI.
This allows me to decide, depending on the use case: Do I need a classic workflow, an agent-based workflow, or a clearly defined agent with access to specific resources? Or do I want an open agent that can utilize the entire knowledge graph and dynamically combine information?
Can specialist users also create such workflows?
I definitely think it makes sense to give these possibilities to specialist users in Joule Studio as well. After all, there are simple processes along the lines of: When a specific email arrives, perform this or that action. Even everyday users can create their own workflows for such tasks — within the rules defined by IT.
What is the next technological step after multi-agent systems?
A trend is already emerging, and initial research results are available: Models are increasingly learning directly how to select and use an agent’s tools — similar to how reasoning abilities were learned about a year and a half ago. This makes the models faster and enables them to make better decisions about which tools to use.
Another very fascinating topic is dynamic learning. The idea is that a model receives feedback and adjusts its own weights accordingly. It actually changes itself. In my estimation, however, research in this area is still in its early stages.
Today, this is often solved using a kind of “second brain”: Information is stored and retrieved as needed. If, however, models could learn directly from feedback, that would, in my view, be a major step towards greater intelligence.
Cybersecurity is also becoming increasingly important. AI is changing both the threat landscape and the possibilities for defense. In my estimation, we will see significantly more specialized models in this area.
And then there are some fascinating approaches, such as that of Yann LeCun, who thinks about how model architectures should work in a completely different way. He sees limits to what LLMs can achieve and therefore pursues other mathematical approaches.
Understanding the world in a different way also means incorporating significantly more data. In a typical business workflow, I might work with 15 tables. That’s a different order of magnitude than a world model — although even that term is defined differently.
LeCun’s approaches, for example, aim to infer missing information in an image or video, rather than predicting the next token. This allows for the inclusion of different and more extensive data because the model doesn’t have to predict every detail, as is the case with a purely autoregressive approach. Which approach is the right one is sometimes debated almost as a matter of faith.