On this pageJump to a section
Automation rarely fails in the middle. It fails at the edge, where it should have handed off and did not.
The middle of a workflow is the part everyone designs: what the tool reads, how it replies, where it files the result. The seam is the part nobody designs: the moment a person is supposed to take over. When that moment is vague, the machine keeps going, and it keeps going confidently. Here is how I think about building the seam on purpose.
What does "human in the loop" actually mean?
It means the human is a designed step, not a rescue.
Anthropic's guide to building agents draws a useful line. In a workflow, the steps are fixed in code ahead of time. In an agent, the model decides its own steps and which tools to use. In Anthropic's description, an agent can stop at checkpoints or when it gets stuck and hand the question back to a person for information or judgment.1
Success in the LLM space isn't about building the most sophisticated system. It's about building the right system for your needs.
Simple first, autonomy later
The same guide says to keep things simple and add complexity only when it demonstrably improves outcomes, and testing extensively in sandboxed environments with appropriate guardrails.1 That is good advice for a small business. A tool that drafts replies and waits for approval is easier to trust than one that sends them on its own, and you can loosen it as it earns that trust. (Agents vs Workflows covers when each fits.)
What happens when the seam is vague?
Two public cases show the shape of the risk.
The chatbot that invented a policy
In a widely discussed Canadian case, a traveler asked an airline's chatbot about bereavement fares. According to the Manatt summary, the bot said he could book a regular ticket and request a partial refund within 90 days. The airline's staff later said retroactive applications were not permitted. The British Columbia Civil Resolution Tribunal rejected the airline's attempt to treat the chatbot as separate from the company, found it did not take reasonable care to ensure its chatbot was accurate, and ordered a partial refund of CAD $650.88 plus interest and fees.2
The lesson is not about airlines. It is that a customer-facing tool speaks for you, which I wrote about in AI Customer Support Without the Air Canada Mistake. If a human reviewed the answer before it reached the customer, or if the bot had been required to quote the actual policy and hand off any exception, the error would have had somewhere to stop.
The shop that gave things away
In Anthropic's Project Vend (see what an AI running a shop teaches about delegation), an AI agent ran a small office shop for about a month. It did some things well, such as finding suppliers. It also gave customers a payment account it had made up, sold items below cost and kept offering discounts after saying it would stop. Anthropic said it would not hire the agent today, but that many failures looked fixable with better scaffolding and oversight.3 Nothing in the setup forced a person to look at pricing or payment instructions before they reached a customer.
Anthropic's own short video on the experiment, posted on December 18, 2025, adds a candid line from the team: "We were poorly calibrated to how bad the agents were at spotting what was weird."4 That is the case for a designed handoff. If the agent cannot tell when something is off, a trigger has to tell it.
VideoAnthropic
Claude ran a business in our office
The Project Vend team explains the early mistakes and why they added a CEO sub-agent.
What triggers should end the AI's turn?
Decide in advance. Write them down.
| Weak handoff | Designed handoff |
|---|---|
| "The AI will escalate if needed" | A written list of triggers that stop the AI automatically |
| The person finds out after the customer complains | The person is notified the moment a trigger fires |
| The human starts from scratch | The human gets a summary, the history and the decision needed |
| Nobody knows who owns the outcome | One named person owns it, with a response time |
Six triggers worth having
- Money moves. Refunds, discounts, price changes, payments, anything that touches a balance.
- A promise or claim is made. Guarantees, timelines, results and policy statements.
- The request is new. If the AI has not seen this kind of question before, it should not guess.
- The person is upset. Frustration, a complaint or legal language goes to a person.
- The action is irreversible. Deleting, sending to a list, publishing, changing access.
- The AI is unsure. Give it permission to say "I'm not certain, let me bring in a person."
What context should the human receive?
Everything that stops them from asking the customer to repeat themselves.
- AI handles the middleReads, replies, files
- A trigger firesMoney, promise, new, upset, irreversible or unsure
- The packet is sentWho, what they asked, what the AI did, why it stopped
- A named owner decidesWithin the promised response time
The handoff packet
- Who: the customer, their account and how they reached you.
- What they asked: the original message in their own words.
- What the AI did or said: every reply and action so far.
- Why it stopped: which trigger fired.
- What decision is needed: a specific question for the human, such as approve this refund or answer this claim.
- By when: the response time you promised.
A handoff without this is a transfer of confusion. The customer feels it immediately.
Who owns the outcome?
One name, not a team.
Every automated flow needs one person who is responsible for what it does (your AI policy is the place to write the name down), including what it does at 2 a.m. If that is you, say so, and decide when you will look: what you review daily, what weekly and what you only see when a trigger fires.
Review the seams monthly
Read a sample of handoffs, plus a sample of conversations that were not handed off. The second sample is where you find the ones that should have been.
What to remember
AI agents do a lot of the building in my one-person business. So the question I keep asking is not "what can it do" but "where does it stop." The handoff is where trust is built, because it shows you built the system to be corrected.
Autonomy is something a system earns one handoff at a time.
Designing the seam is part of how I build agent workflows inside the systems I build for clients.
Keep building.Ekai
Questions people ask
What does human in the loop mean for AI?
It means a person reviews, approves or takes over at defined points in an automated process. Anthropic describes agents that stop at checkpoints, or when stuck, to get a human decision, instead of running to the end unchecked.
When should an AI hand off to a human?
At least when money moves, a claim or promise is made, the request is new or unclear, the person is upset, or an action cannot be undone. Define these triggers in advance, in writing, instead of hoping the model notices.
Is a company responsible for what its chatbot says?
In the Air Canada case, a British Columbia tribunal rejected the airline's argument that it could not be liable for its chatbot's information and ordered a partial refund of CAD $650.88. It is a single decision in one jurisdiction, but it shows the risk.
What should be passed to the human during a handoff?
Everything the person needs to avoid starting over: who the customer is, what they asked, what the AI already said or did, why it stopped and what decision is needed.
Sources
- 1
- 2
- 3
- 4
