Claude 3.5 Computer Use Feature Explained: How Developers Can Make It See the Screen and Click the Mouse

What made this Claude 3.5 update really “go viral” isn’t that it’s better at chatting—it’s that it’s starting to learn how to “use a computer.” The upgraded Claude 3.5 Sonnet strengthens coding ability and also brings a public beta computer use feature: it can see the screen, move the cursor, click buttons, and type text. For people building automation and intelligent assistants, this opens up a very practical new path.

Claude 3.5 Model Upgrade: Sonnet Is Stronger, Haiku Is Faster

Anthropic has simultaneously released the upgraded Claude 3.5 Sonnet, as well as the new Claude 3.5 Haiku. The official statement emphasizes that Claude 3.5 Sonnet’s improvements are “comprehensive,” and especially more prominent in software engineering and coding tasks.

If you care more about speed and cost, Claude 3.5 Haiku focuses on low latency and stronger instruction-following, positioned for user-facing products and sub-agent tasks. Put simply: Claude 3.5 Sonnet leans toward a higher “capability ceiling,” while Claude 3.5 Haiku leans toward “response speed and cost-effectiveness.”

What the Computer Use Feature Is: Letting Claude 3.5 Operate Interfaces Like a Human

Computer use is a public beta capability in Claude 3.5, currently mainly aimed at API scenarios. Its core idea is not to call a fixed tool interface, but to have the model understand the current UI by “looking at the screen,” then perform actions such as moving the cursor, clicking, and keyboard input.

It’s worth noting that Claude 3.5’s computer use is still experimental, and the official documentation also plainly says it can be “a bit finicky and error-prone.” Therefore, it’s better suited for workflows that are reversible and verifiable, rather than one-off, irreversible critical operations.

What Workflows Claude 3.5 Can Automate End-to-End

When a task requires dozens or even hundreds of UI steps, Claude 3.5’s computer use becomes valuable—for example, configuring items one by one in a web admin panel, filling out forms across pages in enterprise tools, or completing lookups and data entry in multi-step ticketing systems. Its advantage is that it can “follow the UI,” so you don’t have to write separate scripts for each page.

Many teams are already exploring similar capabilities, applying Claude 3.5 to longer, end-to-end workflows. In real deployments, it’s recommended to turn key steps into “checkpoints,” having Claude 3.5 output screenshots/state summaries after each segment so humans or programs can verify them.

Availability Channels and Security Info: Don’t Skip Pre-Deployment Evaluation

The upgraded Claude 3.5 Sonnet is now available to users, while the computer use feature is offered as a public beta in the API and can be built on Anthropic API, Amazon Bedrock, and Google Cloud Vertex AI. On the AWS side, you may see it under the name Claude 3.5 Sonnet V2.

On security, the official disclosure says it collaborated with AI safety research institutions in the U.S. and U.K. for pre-deployment testing and considers its ASL-2 standard still applicable. For developers, the more practical advice is: add permission boundaries, operation logs, and failure fallbacks to Claude 3.5’s computer use, and first make sure it’s “controllable, auditable, and stoppable.”