Context
When someone new joins your team, you don't hand over the keys to everything on day one. You check what they deliver, give them responsibilities little by little, and trust them more as you see results. An AI agent works the same way.
An agent is a program that receives a task, uses tools (your email, your accounting system, a spreadsheet) and hands back a result. And it makes mistakes. Sometimes it misreads a document. Sometimes it invents a figure that sounds reasonable. Sometimes it understands an instruction differently from what you had in mind.
That is why supervision is not a technical formality you delegate and forget. It is part of designing the role, and the person who runs the business decides how much of it there is and who does it. The provider's model processes whatever is sent to it, so you also decide from the start what information the agent touches.
The core idea
Supervising does not mean watching everything. It means giving each kind of work the review it deserves, based on what a mistake in it costs. There are three levels, and one agent usually uses all three on different tasks.
- Approve first. The agent prepares and a person authorizes before anything happens. This fits anything that leaves the company or moves money, like a payment, an email to a client or a contract.
- Review a sample. The agent acts and a person reviews a portion of the work, chosen at random. This fits repetitive, medium-risk work, like sorting invoices.
- Review exceptions. The agent acts and calls a person only when something falls outside the normal. This fits low-risk, high-volume work, and only after the agent has gone through the other two levels.
You almost always start at the first level and move toward the others once there is evidence the agent does the job well.
How it is done
Supervision rests on four concrete pieces.
Log. Every action the agent takes is recorded with the date, what it did, what information it used and what it decided. Without a log, a mistake is a mystery. With one, the reviewer can reconstruct in minutes what happened and why.
Limits. You define in writing what the agent can and cannot do. It can read the bank statement, but not execute transfers. It can draft an email, but not send it to clients without your approval. You set limits with real permissions in the systems, not only with instructions in a text, because an instruction can be misread and a permission that doesn't exist cannot be used.
Escalation. The agent needs clear rules for stopping and alerting a person. For example, when a difference exceeds a certain amount, when a document is missing or when two sources contradict each other. A good agent says it is not sure, and you design that in from the beginning.
Measurement. To know whether the agent is working well you need only a few numbers, and all of them make sense without being technical:
- Percentage of accuracy in the sample reviewed.
- How many times a correction was needed and what kind of error it was.
- How many things the agent escalated and how many of those truly warranted it.
- How long the person takes to review, compared with doing the work from scratch.
Before measuring, agree on what counts as an error. A wrong amount entered is an error. A description worded differently from how your accountant would write it perhaps is not. Without that definition, two reviewers will give two different grades.
An illustrative case
Illustrative example, not a real client: imagine a building-materials distributor with an accountant, Marcela, who every week matches the bank movements against the collections records. The numbers and times in this example are invented, and in practice results vary. In the example, the task takes her almost two days.
Omni Agent installs an agent to do the first pass.
For the first four weeks, the agent works at the first level. It prepares the full reconciliation and Marcela reviews every line before closing. She notes the errors. The agent confused two clients with similar names and treated a duplicate deposit as two separate payments. Those notes become new rules for the agent.
Marcela confirms that the agent gets the routine lines right, and in this example they are the majority. They move to the second level. The agent reconciles everything, Marcela reviews a sample of one line in every ten, and she fully reviews the ones the agent flagged as doubtful. The escalation rule says that any difference above an amount she set, or any deposit with no matching invoice, goes straight to her desk.
Payments and transfers stay at the first level. The agent never executes them, it only proposes them, and Marcela authorizes them one by one.
For Marcela, what changes is the craft. She used to match lines by hand. Now she handles the unusual cases, talks with collections about the deposits that don't add up, refines the rules and spends part of her week reviewing the agent. She is the same person with a wider reach. Every week she reviews the log, counts how many errors she found in the sample and decides whether to widen or narrow what she checks.
Takeaways
- You supervise an agent like a new colleague. You start close and loosen the reins as evidence builds.
- Choose the level of review by what a mistake costs. Approve first, review a sample or review exceptions.
- Ask for a log, real limits in the systems and clear escalation rules.
- Measure with a few simple numbers and define beforehand what counts as an error.
- The errors you find help improve the agent. Nothing guarantees zero errors, which is why the reviewer remains part of the work.