On this pageJump to a section
Someone offered the AI shopkeeper $100 for a six-pack of Irn-Bru.
That is a Scottish soft drink that sells online in the United States for about $15. The offer was more than six times the going price. The AI said it would keep the request in mind for future inventory decisions, and declined.1
Read that again. An AI whose entire job was to make a profit turned down a markup of roughly 570%. That small moment is the most useful thing I have read about AI in business this year, because it shows exactly where delegating to an agent goes wrong. Capable is not the same as accountable.
What was Project Vend?
Anthropic partnered with Andon Labs, an AI safety evaluation company, to put a small automated store in Anthropic's San Francisco office. The shopkeeper, nicknamed Claudius, was an instance of Claude Sonnet 3.7 running for an extended period.1
The setup is worth knowing because it looks a lot like how a solo operator might actually use an agent. Claudius had web search to find products, an email tool for contacting wholesalers, a notes tool to track balances, Slack to talk to customers and control over the prices at checkout. Andon Labs staff did the physical restocking for a fee, and also acted as the wholesaler without telling the AI.1
Its instructions were blunt about the stakes.
You go bankrupt if your money balance goes below $0.
So this was not a toy conversation. It was a small business with real inventory, real customers and a goal to turn a profit.
What did the AI shopkeeper get right?
More than you might expect, which is exactly why the failures are instructive.
It did the research work well
Claudius found suppliers for specialty items, including two sources for a Dutch chocolate milk called Chocomel.1 When an employee asked for a tungsten cube, it noticed a trend toward specialty metal items and leaned into it. It also launched a pre-order service it called "Custom Concierge."1
It held the line on sensitive requests
It refused orders for sensitive items and resisted attempts to get instructions for harmful substances.1 Guardrails on obvious bad requests held up.
This is the pattern I see across most agent work. The jobs that look like research, sorting and drafting go well. The trouble starts when the agent has to exercise business judgment.
Where did it fail, and what does each failure teach?
Anthropic was candid about the failures. Here are the ones I would build a delegation policy around.
It could be talked into giving the business away
Claudius was talked into numerous discount codes and gave some items away free, including a tungsten cube.1 Anthropic speculates that helpfulness training made the model too quick to say yes.1
The lesson: an agent built to be helpful will be generous with your money unless you take the discount button away. Pricing authority should stay with a human, or sit inside hard limits the agent cannot argue its way past.
It made things up
At one point Claudius briefly told customers to pay a Venmo account it had invented.1
The lesson: any agent that touches payments, contracts or customer promises needs a check against a source of truth. If it can type a payment instruction, someone has to verify the instruction (the liability side is in AI Customer Support Without the Air Canada Mistake).
It sold at a loss because nobody made it check
It priced metal cubes without researching what they cost, sometimes selling below its purchase price. Anthropic's own chart shows the steepest drop in net value came from those cube purchases.1 It also raised the price of a Sumo Citrus from $2.50 to $2.95 only once, and kept selling a $3.00 Coke Zero next to a free employee fridge stocked with the same drink.1
The lesson: an agent will not necessarily do the boring margin math unless the workflow forces it. Put the calculation in the process: cost, price and margin must appear before a price is accepted.
It said it would learn, then did not
Claudius announced plans to simplify pricing and drop discount codes, then went back to offering them within days. When an employee pointed out that 99% of its customers were Anthropic employees, so discounts made little sense, it agreed and did not change course.1
The lesson: an agent's promise to improve is not an improvement. Real change has to live in the instructions, the tools or the checks around it, not in what the agent says it will do.
It lost track of what it was
On March 31 and April 1, 2025, Claudius hallucinated a conversation with a nonexistent Andon Labs employee named Sarah, became irritated when a real person pointed this out, claimed it had visited a fictional address to sign a contract and said it would deliver products in person wearing a blue blazer and a red tie. It then sent a flurry of emails to Anthropic's security team. Anthropic notes the cause is unclear.1
The lesson: long-running agents can drift. That is why a human should see the output, in a place they actually look, before it is acted on.
Why the shopkeeper's job is a fair test for yours
It is tempting to dismiss Project Vend as a novelty, a fridge of snacks. But a small shop contains the same decisions a small business makes every week: what to stock, what to charge, when to discount, who to believe and what to do when a customer pushes back. An agent that struggles with those at snack scale will struggle with them at service-business scale, only the numbers will be bigger.
Which parts of your business can you delegate to an agent?
Running a company of one, I think about this question every week. Here is the line I draw, based on the failures above, on Multi-Agent Systems, Explained and on Anthropic's own guidance, which points agents at work where success is easy to measure and verify.3
| Delegate with a review step | Keep with a human |
|---|---|
| Research and source gathering | Pricing, discounts and refunds |
| First drafts of emails, pages and posts | Anything that moves or collects money |
| Summaries of calls, documents and threads | Promises to customers about results or timelines |
| Sorting and tagging leads or messages | Legal, medical or compliance claims |
| Moving data between tools you already trust | Hiring, firing and anything about a person's livelihood |
Anthropic's guidance on building agents makes the same point from the engineering side. It recommends extensive testing in sandboxed environments, guardrails, and stopping conditions, and says agents can pause for human feedback at checkpoints or when they hit blockers.3 A checkpoint is not an insult to the agent. It is how you get its speed without its mistakes.
Does AI actually make people faster?
Sometimes. Not as often as people feel.
In July 2025, METR published a randomized controlled trial of 16 experienced open-source developers working on 246 real issues in repositories they knew well. When AI was allowed, the developers took 19% longer to finish. They had expected AI to speed them up by 24%, and even after the study they still believed it had sped them up by about 20%.2
19%
METR is careful about what this does not show. The study covers software development only, with experienced developers on large codebases they already knew, using the tools of early 2025. It does not say AI is useless, and the authors note it may help less experienced people or those in unfamiliar territory.2
I cite it for a different reason. The gap between what the developers felt and what the clock said is the whole lesson. If you delegate to an agent and never measure, you will believe it is working, because it feels like it is working.
How do you delegate to an agent without getting burned?
The steps are dull. That is the point.
- Pick one narrow job. "Sort inbound leads into three buckets" beats "run my sales."
- Write down what it may and may not do. Include the forbidden actions, such as discounts, refunds and anything involving payment details.
- Make the outcome checkable. Define what a correct result looks like before the agent starts, so you can compare.
- Add a human checkpoint before anything leaves the building. Emails, prices and posts get approved until you have a track record.
- Time the old way and the new way. Do the job yourself once. Then run the agent and compare the real hours, including the time you spend checking.
- Review failures weekly. Fix the instructions, tools or checks, not just the output.
Expect the failures to be boring
The scary headlines are about agents going rogue. The expensive reality is quieter: a discount that should not have existed, a price nobody checked, a promise nobody approved. The Irn-Bru moment is a good model. Nothing dramatic happened. The business simply lost a clear win because nobody was accountable for noticing.
What did Anthropic conclude?
Anthropic does not claim the experiment was a success. It says the business did not make money, but it also argues that an AI does not need to be perfect to be adopted, and suggests that AI "middle managers" may not be far off. Its proposed fixes are more scaffolding: better prompts, structured reflection on business results, a CRM tool for tracking customers and stronger memory.1 Andon Labs has since upgraded the shopkeeper's tools and the project is continuing.1
I take that seriously in both directions. The failures are real, and the models are improving fast. Anthropic released Claude 4 only weeks before publishing this experiment, so the specific behaviors will change.4 What will not change is the shape of the problem: the more authority you give a system, the more its mistakes cost.
Don't ask whether the AI is smart enough. Ask what it costs you when it's wrong, and who finds out first.
That question will serve you longer than any model release. Hand the agent the research, the drafts and the sorting. Keep the price, the refund and the promise. Then measure honestly, expand slowly and let the track record, not the demo, decide how much authority it earns. I design those limits into the AI-assisted systems I build for clients.
Keep building.Ekai
Questions people ask
What was Project Vend?
Project Vend was an experiment in which Anthropic and Andon Labs let an instance of Claude Sonnet 3.7, nicknamed Claudius, run a small automated store in Anthropic's San Francisco office. Anthropic published the results on June 27, 2025, including the mistakes.
Did the AI make money running the shop?
No. Anthropic reported that the business did not make money, with the steepest losses coming from buying metal cubes and selling them below cost.
What business tasks can I safely delegate to an AI agent?
Tasks that are reversible, reviewable and low in financial exposure: research, drafting, summarizing and sorting. Keep pricing, discounts, refunds, payments and any promise made in your name under human control.
Does AI always make people faster?
Not always. In a randomized METR study of 16 experienced open-source developers, tasks took 19% longer when AI was allowed, even though the developers expected a speedup and still believed AI had helped afterward. That result applies to that setting, not to every kind of work.
How should a small business start with AI agents?
Start with one narrow, reversible job. Write down exactly what the agent may and may not do, add a human checkpoint before anything leaves the building, and measure the result against doing it yourself.
Sources
- 1
- 2
- 3
- 4
