Anthropic has recently delivered a major update to Claude 3.5 Sonnet—far beyond a simple patch. The AI now has the ability to directly control a computer. Simply put, Claude can see your screen, move the mouse, and even type. Meanwhile, the all-new Claude 3.5 Haiku model launches at blazing speed, posting standout scores across multiple benchmarks. This upgrade unlocks new creative potential for developers and regular users alike.
Claude 3.5 Sonnet’s New Skill: Using a Computer Like a Human
The headline feature of this update is Claude 3.5 Sonnet’s newly acquired computer control capability. Anthropic built a dedicated API that lets Claude perceive pixel-level screen information and interact with it just like a person. Developers can integrate this API to enable Claude to perform multi-step tasks such as “open the browser, log into email, and send a message.” In the OSWorld benchmark, Claude 3.5 Sonnet achieved a 14.9% score using only screenshots—far ahead of the runner-up.
That said, the feature is still in testing. Claude struggles with fine-grained actions like scrolling and dragging—for example, it may interrupt tasks during long screen recordings. However, companies including Canva and DoorDash are already testing this Claude function to automate repetitive workflows. Replit, for instance, uses it to build an agent that can automatically inspect applications.
Coding Performance Leap: Industry-Leading Programming Capabilities
Beyond computer control, Claude 3.5 Sonnet’s coding skills have improved dramatically. Its score on the SWE-bench Verified test jumped from 33.4% to 49.0%, surpassing many public models, including several specialized reasoning systems. GitLab’s testing shows that the updated Claude offers stronger reasoning for DevSecOps tasks without added latency.
