AI governance is evolving from periodic compliance checks to a vital component integrated into an organization’s design and operationalized in real time. As autonomous agents execute tasks instantly, the gap between decisions and outcomes narrows, pushing governance into daily operations.
“Traditional strategic governance, applied to AI as you would with applications and systems, doesn’t work for AI agents,” says Philipp Herzig, CTO of SAP. “Autonomy accelerates events. Agents act on your behalf, sometimes without explicit approval. Proactive real-time operational governance prevents issues rather than addressing them after the fact.”
Regulated industries lead the charge in AI governance
Continuous AI governance requires enterprises to answer four questions at all times:
- Which AI agents exist across the enterprise, and what purpose does each serve?
- What data and systems can each agent access?
- How does each agent participate in business processes?
- Is each agent operating within established policy?
Financial services, healthcare, pharmaceutical, and public sector organizations face the greatest urgency around these questions, as regulators already demand documented accountability. The AI governance capabilities these organizations develop will likely become standard in other industries.
The limitations of traditional technology governance become evident in regulated industries when AI agents operate within existing accountability and compliance frameworks. Banks apply model risk management guidance such as SR 11-7 and SR 26-2. Drug manufacturers adhere to GxP compliance and FDA requirements. Government agencies comply with FedRAMP, authority to operate, and data sovereignty rules. Applying these regimes to non-deterministic systems creates expectations most enterprises cannot yet meet.
Regulators require organizations to reconstruct any decision an AI system made at any point. “Consider the pharmaceutical industry, with its chain of custody,” Herzig explains. “If you manufacture drugs, you must track every point where the product was handled. Blank spots in a regulated process are unacceptable.”
Meeting industry standards requires more than a session log. Ownership must be tied to the agent, the workflow it participates in, and the guardrails and delegated authority it holds in a user role. Identity and access management, traditionally focused on humans, must now account for entities that independently pursue assigned goals, including finding ways around API restrictions.
Permissions granted at deployment can diverge from an agent’s actual usage over time, creating a gap between its authorized role and real-world behavior. Continuous monitoring is essential to ensure access remains appropriate as the agent’s activity evolves.
AI inventory and agent discovery are key to control
Effective AI governance begins with a current inventory of the AI estate, which most enterprises lack, according to Herzig. “You can’t manage what you can’t see. Without automatic discovery and maintenance of an AI asset or agent inventory, governance becomes impossible.”
Discovery must be continuous, as creation and deployment occur constantly. An employee with a chat interface can deploy a functioning agent in an afternoon, immediately impacting real work.
“Imagine someone builds an agent to handle invoices using a copilot tool, handing over the task,” Herzig says. “Now, something operates on finances and customer relationships without oversight. Shadow IT once meant unauthorized software on a laptop. This version is like an employee bringing three unseen coworkers, assigning them tasks, and never informing the boss.”
The governance perimeter extends beyond agents. Large language models, MCP servers interfacing with applications, and agent-to-agent protocols fall within its scope. Multi-agent orchestration adds complexity, making responsibility assignment challenging when agents delegate tasks. Herzig estimates agents account for less than a third of what an enterprise must manage.
Contextualizing AI within the broader business and architecture is vital to understand dependencies and fully map risk. This transparency enables holistic AI governance.
Related Post: Databricks agents match top AI answer quality in half
An inventory identifies AI assets, but effective governance also requires understanding their position in the enterprise architecture and the business processes they support. Mapping these relationships reveals dependencies and potential impacts when issues arise.
“Maintaining a registry is one thing; positioning the estate in the business context is another,” Herzig notes. “Knowing which agents operate within which processes and which applications rely on them answers the harder questions.”
Runtime AI governance ensures policies hold during agent execution. The EU AI Act and the NIST AI Risk Management Framework raise standards for documentation, risk classification, and human oversight, but checklists alone are insufficient without enforcement.
SAP’s approach provides visibility into existing AI risks across the enterprise and extends control to runtime management. “Agents learn, adjust, and drift,” Herzig says. “Without runtime measurement and control, AI governance is ineffective. Governance without enforcement is like a governor without courts, law enforcement, or jails.”
Reconstructing an agent’s actions after accessing a restricted system leaves cleanup to process owners, so enforcement must occur in real time. Breaking down silos among enterprise architects, CISOs, and compliance teams is essential, driving the rise of AI centers of excellence in large organizations.
AI governance data also measures agent ROI and process performance. Linking agent execution to process conformance shows whether agents operate as intended and improve supported business processes. This transforms governance data into evidence of performance and ROI, not just risk avoidance.
Process benchmarks provide metrics for deployments, aggregating at the process or business capability level. “Agents generate telemetry like token input, model usage, tool call health, and success rates,” Herzig explains. “The challenge is aggregating this data against processes previously run by humans. Proving efficiency gains requires tools that roll up signals to the business value level.”
SAP’s AI governance approach across architecture, identity, processes, and compliance
SAP connects these domains through products offering different governance layers. SAP LeanIX provides architectural and compliance context, SAP Signavio offers business process context and value measurement, Cloud Identity Services handles identity, and SuccessFactors provides workforce insights. The AI Agent Hub serves as the central entry point.
Discovery capabilities identify and inventory agents across the enterprise, including those outside SAP. Verification gates in the deployment path ensure MCP servers and agents from Joule Studio meet standards before production use.
SAP plans to align verification with regulations like the EU AI Act and introduce runtime enforcement capabilities by 2026.
AmTrust Financial Services expanded its SAP LeanIX enterprise architecture practice to AI governance ahead of EU AI Act requirements, creating an AI inventory, tagging applications by risk, and forming a cross-functional AI governance council. CapitaLand developed reusable governance frameworks across business units in over 260 cities, enabling governance to scale with adoption.
The greater challenge lies between vendors, Herzig notes. “Our discovery covers ServiceNow, AWS, Microsoft, and Google agents, beyond SAP. The industry needs an open agent ecosystem. Granular telemetry for performance monitoring and business outcome aggregation remains siloed in each tech environment. No organization relies solely on one vendor. Opening this up improves AI governance for all.”
