Anthropic releases AI to automate mouse clicks for coders

SAN FRANCISCO – Anthropic, a startup backed by Alphabet and Amazon.com, released a pair of updated artificial intelligence models on Tuesday, along with a new capability to autonomously perform computer tasks and save users keystrokes.
The new “computer use” feature can tell AI “where to move the mouse, where to click, what to type, in order to do quite complicated tasks,” Anthropic’s Chief Science Officer Jared Kaplan said in an interview.
The capability is tailored to software developers and represents a move toward AI agents, programs that require little human intervention to carry out multi-step actions. Researchers have touted agents as a frontier for AI development beyond chatbots, which easily conjure prose or computer code though not actions.
Anthropic demonstrated a use case for the feature that entailed coding a basic website, and another that used various programs including Google Search and Apple Maps to plan a sunrise outing.
Anthropic offers software developers three versions of Claude, its family of AI models, at price points that vary based on their performance. This week’s updates come to Sonnet, the mid-tier model, and Haiku, the cheapest.
The new 3.5 Haiku can generate computer code in a manner “almost comparable” to the version of Sonnet released in June, according to Kaplan. CEO Dario Amodei told Reuters at the time that the company intended to update Opus, the most capable model, by the end of the year.
The computer use feature is currently limited to the new version of Claude 3.5 Sonnet and comes with safeguards to prevent its application toward spam, fraud and election-related misuse, Anthropic said. Kaplan said the AI still makes mistakes.
Mike Krieger, a co-founder of Instagram who joined Anthropic this spring as chief product officer, said the company wants feedback from business customers to learn where to focus development of the feature. Meanwhile, a labs team inside Anthropic is exploring how to make the capability available for consumers, something Krieger said he personally wants.
“I was booking flights,” he said. “I really just want this to be completely automated.”
Microsoft on Monday unveiled an application for its clients to build their own agents that can handle queries, identify sales leads and manage inventory.

(Reporting By Kenrick Cai and Jeffrey Dastin in San Francisco; Editing by Lincoln Feast.)