Let’s start with some good news: if you’re planning how to integrate agentic AI into your enterprise, you don’t need to throw out existing AI governance frameworks and spend vast sums of money starting over.

In my view, the right approach is to build on the governance we already have, considering that autonomy demands real guardrails such as strict service credentials, execution rules, and automated kill switches for when things go off the rails. 

Most Chief Data Officers (CDOs) will have seen their governance progress from traditional model risk management to AI governance and, more recently, generative AI governance. 

If Agentic AI is your next step, the foundation remains the same, but the risks change when AI moves from providing a response to actually taking action.

Traditional generative AI has largely centered on a person asking a question, receiving a response, and deciding what to do with it. 

With agentic AI, the system can plan, call different tools or systems, and execute multiple steps autonomously. Failures snowball fast. A bad assumption early on quietly infects every action that follows. If the agent moves money on step two and updates records on step three, hitting the brakes when it crashes on step four is too little, too late. Without a way to automatically roll back those earlier actions, you are left with a real operational nightmare.  

An agent is not necessarily agentic AI

There is still some confusion in the industry about the difference between an agent and agentic AI.

An agent might read a report using predefined prompts and provide a response for someone to review. It is performing a specific task, and existing AI governance frameworks may be perfectly capable of managing that use case.

A truly agentic AI process involves multiple steps and the ability to plan and take action using different tools or systems. 

For example, I work for an HR benefits organization. Imagine someone wants to transfer a 401(k) balance from one organization to another. 

An agentic process might first verify the individual, determine what needs to happen, conduct the transaction, identify where the funds need to move, confirm completion, and send the appropriate communications.

Now we are no longer governing one response or one task. The AI is executing a process across multiple steps and potentially multiple systems. We have to think about access controls, system execution, API calls, transaction limits, and what happens if one action leads to another we did not intend.

A limited agent performing a single task may fit within an existing AI governance framework. When we’re talking about agentic AI with multiple steps and capabilities, we have additional risks to consider. 

Treat your agent like a junior employee

One way I think about agentic AI is to treat an agent almost like a junior employee, with one critical difference: unlike an employee, an agent lacks human judgment and can be tricked by malicious actors.

Would you give a junior employee access to every system in your organization on day one? Would you immediately give that person full read and edit access and allow them to execute any transaction?

Probably not.

You would start with the access required to do the job. You might begin with read-only access, make sure the employee understands the work, and expand permissions as appropriate.

We can take a similar approach with agents. Start with limited access. Test the agent in a sandbox and production environment. Make sure it continues to execute as designed before expanding what it is allowed to do.

Traditional AI governance can still do a lot of the work. We don’t need to reinvent the entire wheel. We need to understand what additional capabilities we are giving the agent, what additional risks those capabilities create, and what controls need to be added.

Quality by design means defining what an agent cannot do

I frequently advocate for “Quality by Design”, and I think it becomes even more important with agentic AI. Most development teams focus on the “happy path”, engineering for what the agent should do. 

But true “Quality by Design” requires being intentional about negative constraints: architecting what the agent is not permitted to touch, trigger, access, or answer before anyone even starts coding.

We often tell developers what we want an agent to do. We define the use case, test the agent against 100 scenarios, see that those scenarios work, and move the agent into production.

But did we tell it what it shouldn’t do?

Suppose we build an agent for a specific benefits transaction. We test the questions and actions related to that transaction, and everything works exactly as expected. Then someone asks the agent a question about an unrelated health issue or a loan.

If we never defined that boundary, the AI may try to answer.

Before developers begin coding an agent, organizations should define the questions, topics, and actions the agent is allowed to handle. We should be just as clear about what it cannot handle and when it should stop, decline a request, or direct someone to another channel.

Undefined boundaries become a bigger concern with agentic AI because a response can lead directly to another action. 

If an agent is allowed to choose its own path when we intend it to choose between Path A and Path B, we may get a completely different outcome than the one we designed for. Defining those boundaries before an agent goes into production is part of quality by design. 

How much autonomy is too much?

I don’t think anyone can give every organization the same answer on how much control to give agentic AI.

The right boundary depends on what the business is trying to accomplish, how the AI will be used, and the regulatory and compliance requirements that apply. 

In our organization, we view AI as a force multiplier that can help people with life-critical decisions. We don’t want AI making those critical life decisions for them. At the same time, transactions such as moving a 401(k) balance or enrolling someone in a particular plan may be appropriate for AI.

That boundary will be different for another company, and it will probably change over time. Five years from now, after organizations have more experience with agentic systems, we may be comfortable allowing AI to do things we would not allow today. In my view, though, there will always be limits on what AI and agents should be allowed to do.

Putting a human in the loop does not solve everything

Human-in-the-loop and human-on-the-loop controls can help organizations manage the level of autonomy given to AI agents. 

• Human-in-the-loop requires a person to review or approve an action at a defined point before the agent can continue. 
• Human-on-the-loop keeps a person involved through oversight of the agent’s activity or review of what it has completed.  

Take a financial transaction. An agent might be allowed to process the transaction until it reaches a defined dollar threshold. Above that threshold, a person has to review and approve it before the agent can continue.

A human-on-the-loop approach is slightly different. The agent can continue the process while a person oversees its work or reviews what has been completed to determine whether it was done correctly. 

Both approaches can be useful. But if you ask someone to review the same transactions every day and the AI gets them right over and over again, what happens after the hundredth successful transaction? At some point, the person may start approving the AI’s work without really reviewing it.

We cannot treat “human in the loop” as a policy statement and assume the risk has been addressed. If a high-risk control relies on an analyst clicking “Approve” 200 times before lunch, you do not have a governance control; you have an ergonomic hazard.

When the AI agent gets things right the first 99 times in a row, human nature takes over. People tune out, get another coffee, turn the approval queue into a high-speed game of compliance “Whac-A-Mole.” To keep human oversight meaningful, organizations need to build a system that fights rubber-stamping. 

• Reward the catch, not the click: Borrow from cybersecurity bug bounty programs. When an analyst identifies an agent edge case, hallucinations, or flawed chain-of-logic that prompts a model retrain or rule update, they log a “catch”
• Borrow the TSA “golden ticket”: Airport security periodically sneaks fake test bags through scanners to keep screeners alert. Do the same by dropping deliberate, synthetic agent slip-ups into the queue, and recognize reviewers who catch them.
• Unlockable clearance levels: Treat permissions like leveling up in a game. Reviewers who prove they actually scrutinize cases, rather than speed-clicking through, unlock higher tiers, get assigned the higher-value transactions, and work directly with engineers to tune agents’ prompts and guardrails.

Stop putting agents on bad processes

One of the biggest mistakes I see is putting agents on existing processes without first asking whether those processes should exist. 

Imagine an organization has a legacy system and someone has been doing a manual reconciliation for years. The immediate reaction may be: ‘let’s give that person an agent’. The agent can do the reconciliation instead.

But why is anyone doing the manual reconciliation in the first place?

Maybe we should automate or redesign the underlying process so the reconciliation disappears altogether.

If the agent fails, we may conclude that agentic AI didn’t work. But did the technology fail, or did we put it into an environment where it never should have been used?

Legacy organizations face a different strategic challenge because of the systems and processes they already have in place. Before deciding where to add an agent, organizations should:

• Review the process end to end
• Understand why it works the way it does today
• Decide what they are actually trying to accomplish

The governance fundamentals do not necessarily change. Access controls, cybersecurity, data and privacy controls still apply. What changes is the strategy for where agentic AI belongs.

Don’t wait for perfect data

The same data governance principles that mattered before generative and agentic AI still matter now. Organizations still need to:

• Know where their data is located
• Use metadata and lineage to understand where it came from and how it changed across systems
• Apply data quality rules
• Maintain appropriate access controls

These aren’t new concepts. What is new is the magnitude and speed at which AI can expose problems when those foundations are weak.

At the same time, I don’t advocate waiting until every piece of enterprise data is perfect before experimenting with agentic AI. I have previously described the relationship between data and AI governance as a 51/49 split, and the bridge between them is a risk-based approach.  There is a world of difference between an agent that merely reads data to draft a response and one authorized to move money.  The moment an agent acts, the impact shifts from an inaccurate answer to direct operational risk. 

Focus first on the data that is critical to the use case. Make sure the data the agent actually needs is reliable and appropriately governed rather than trying to fix every data problem across the enterprise before you begin.

Agentic AI governance builds on traditional AI governance  

Agentic AI adds new risks, but it does not erase everything organizations already know about governance.

Start with the framework you have:

• Understand whether you are dealing with a limited agent or a truly agentic process 
• Define what the agent can and cannot do
• Limit access based on the work it needs to perform
• Decide where human judgment belongs and make sure that oversight remains meaningful

Before you put an agent on an existing process, ask whether that is a process you should be carrying into the future at all.