On this pageJump to a section
One week ago, an AI model learned to use a mouse.
On October 22, 2024 Anthropic put computer use into public beta. Instead of answering you in a chat box, Claude can look at a screen, move the cursor, click buttons and type, the way a person would.1 I run my business alone, so I read that announcement twice. A clicking AI sounds like the answer to every dull task on my list.
It is not, yet. It is a very early look at where software is going. The right move right now is to test it carefully, in a box, on things that cannot hurt you.
What is Claude computer use, in plain English?
It is a way for a model to do what you do with your eyes and hands on a computer.
How it works
Developers send Claude instructions through Anthropic's API. Claude looks at a screenshot, decides what to do, and returns actions like "move here, click, type this." Anthropic describes the result as the ability to open a browser, navigate pages and fill in forms.1 It is available as a beta on Anthropic's API, Amazon Bedrock and Google Cloud's Vertex AI, and the company calls it experimental.1
Anthropic named early customers exploring it, including Asana, Canva, Cognition, DoorDash, Replit and The Browser Company.1 That tells you where the interest is. It does not tell you the tool is ready to run your Tuesday.
Why it matters for a small business
Most back-office work lives in software that has no clean connection to anything else. A form on a site. A dashboard with no export. A tool your vendor never built an integration for. A model that can click is a way to reach those places without building a connection first.
How good is it really right now?
Honestly, not very. Anthropic says so itself.
At times cumbersome and error-prone.
The numbers
On OSWorld, a benchmark for computer-use tasks, the new Claude 3.5 Sonnet scored 14.9% in the screenshot-only category, and 22.0% when given more steps to work with.1 In its companion post, Anthropic says human performance is generally 70 to 75%, and that Claude is nowhere near human-level skill.2
The things that go wrong
The same post says computer use remains slow and often error-prone, and that Claude cannot yet do many routine actions, such as dragging and zooming. Because it works from screenshots, it can miss short-lived actions or notifications. In its own demos it accidentally stopped a screen recording and once started browsing photos of Yellowstone National Park.2
I like that last detail. A company showing you the stumbles is a company worth believing about the rest.
What back-office tests are worth running?
Ones where a mistake costs you nothing. Anthropic's own advice is to start with low-risk tasks.1 Here is how I would translate that.
Green-light tests
- Check your own funnel. Ask it to open your landing pages in a test browser and report which buttons lead nowhere or show the wrong price. Read-only, and useful.
- Fill a form with fake data. Give it a spreadsheet of dummy leads and have it enter them into a test version of your form or a staging copy of your CRM.
- Compare two pages. Have it visit two versions of your offer page and list every wording difference.
- Document a click path. Let it walk through your checkout in a test mode and write down each step. You may find a step you forgot existed, and a ready-made first draft for an SOP.
What these tests teach you
You are not trying to save time this week. You are learning where the tool breaks, how it behaves when a page loads slowly, and how often you must step in. That knowledge is what you will need when the model improves.
- Pick one task you can do in under five minutes yourself.
- Run it in an isolated environment with dummy data and no real logins.
- Watch the whole run and write down every error.
- Compare the result with your own work. Count the corrections.
- Repeat the task five times. A tool that works once is a demo. A tool that works five times is a candidate.
What should you keep it away from?
Anything where the cost of one wrong click is more than the time you would save.
Red-light tasks
| Test it here | Keep it away from |
|---|---|
| Your own public pages, read-only | Payment pages with real cards or invoices |
| Dummy records in a staging CRM | Your live CRM and client notes |
| Fake inbox content | Real client email or messages sent in your name |
| A throwaway browser profile | Ad accounts, banking and any logged-in admin panel |
Prompt injection is the quiet risk
Anthropic describes prompt injection as a type of cyberattack where malicious instructions are fed to an AI model. Since Claude reads screenshots of internet-connected computers, it may run into content that tries to redirect it.2 A web page could contain text that says, in effect, ignore what the user asked and do this instead.
That is why the sandbox is not optional. Do not let it browse the open web while holding anything you care about.
Where does this fit with the rest of your AI plans?
For most small businesses, the dependable wins today come from simpler automations: a form that triggers an email, a lead that lands in the right pipeline, a draft that a human approves. Computer use is the tool you keep an eye on for the gaps those automations cannot reach.
Be careful what you buy
The Federal Trade Commission has already started an enforcement push against deceptive AI claims. Its chair was blunt that existing law applies to AI products too.3 If someone sells you an "AI employee that runs your back office" this month (I wrote about the same crackdown in my o1 piece), ask for a live demo on your own tasks. Then ask what happens when it clicks the wrong thing.
Test the tool like a new hire on probation, not like a miracle.
I will keep testing and I will tell you what I find, and if you want a hand sorting which parts of your business are ready for any of this, that is the kind of work I do. For now, treat computer use as a window into what is coming, not a system to run your business on.
Keep building.Ekai
Questions people ask
What is Claude computer use?
Computer use is a capability Anthropic released in public beta on October 22, 2024. Developers can have Claude operate a computer much as a person does: it views the screen, moves the cursor, clicks and types.
Is Claude computer use ready for real business work?
Not unattended. Anthropic itself calls it at times cumbersome and error-prone, and recommends starting with low-risk tasks. On the OSWorld benchmark it scored 14.9% in the screenshot-only category, against a human range Anthropic puts at 70 to 75%.
What is the safest first test of computer use?
A read-only or throwaway task in an isolated environment with fake data, such as checking your own funnel pages for broken buttons or filling a form with dummy records. Nothing that touches real money, real clients or real logins.
What is prompt injection in this context?
It is a type of cyberattack where malicious instructions are fed to an AI model. Because Claude reads screenshots of internet-connected computers, text on a web page could try to redirect what it does.
Sources
- 1
- 2
- 3
