Anthropic has introduced a brand-new “Computer Use” feature for its AI code agent tool Claude Code, opening it up to macOS users in the form of a Research Preview. With this feature, Claude can directly control the user’s computer screen—open apps, click buttons, input text, take screenshots—and connect the full workflow in a single conversation, from writing code to UI testing, without leaving the terminal.
What can Claude help you “click”?
According to the official technical documentation, Computer Use is mainly designed for tasks that require a graphical user interface (GUI)—things you normally have to put down the terminal to do yourself:
Build and verify native apps: Ask Claude to write a macOS menu bar app—it will compile it, launch it, and click through every control item to confirm the functionality is working, even before you’ve opened that app.
End-to-end UI testing: Point at a local Electron app and say “test the new user onboarding flow.” Claude will open the app, click through the registration steps, and take screenshots of each screen. No Playwright setup, no test framework.
Debug visual and layout issues: Tell Claude “this Modal gets cut off in a small window,” and it will shrink the window to reproduce the bug, take screenshots, modify the CSS, and then verify the fix again—it can see the screen you see.
Tools that only have a GUI: design tools, hardware control panels, the iOS Simulator, or any proprietary software that has no CLI or API.
Claude’s tool selection priority
Computer Use is the “widest but slowest” tool, and Claude will choose the most precise option based on priority:
If there’s a corresponding MCP Server → use MCP
If it’s a shell command → use Bash
If it’s browser operations and Claude in Chrome is set → use the Chrome extension
If none of the above apply → then enable Computer Use
That is, screen control is reserved for scenarios where other approaches can’t reach: native apps, emulators, and tools without an API.
How to enable Computer Use
Computer Use is provided in the form of a built-in MCP Server (name:
computer-use
). It’s disabled by default and must be manually enabled:
In an interactive Claude Code session, run /mcp, find computer-use in the list (shown as disabled)
After selecting it, click Enable. The settings are saved persistently per project, and you only need to set it once per project
When you first let Claude control your computer, macOS will ask you to grant two permissions: Accessibility and Screen Recording. After granting, choose “Try again.” In some cases, you may need to restart Claude Code
Each session requires authorizing apps one by one
Enable
computer-use
does not mean Claude can control all apps on your computer. In each session, the first time Claude needs to control a particular app, the terminal will show a prompt displaying:
Which apps Claude wants to control
Whether it’s requesting any additional permissions (such as clipboard access)
How many other apps will be hidden during the work
Choose “Allow for this session” or “Deny.” The authorization is only valid for the current session. For apps with broad system access, the system will also show an additional warning:
How Claude will work on your screen
Full-machine exclusive lock: After Computer Use starts, it will acquire an exclusive lock at the machine level. If another Claude Code session is already using the computer, the new request will fail with a message to end the other session first.
Other apps temporarily hidden: When Claude starts controlling the screen, any visible apps that have not been authorized will be hidden, ensuring Claude only interacts with authorized apps. Your terminal window remains visible and won’t be screenshotted, so you can monitor in real time—and Claude can’t see its own output. After Claude finishes, the hidden apps are automatically restored.
Interrupt anytime: After Claude gets the lock, macOS will show a notification “Claude is using your computer · press Esc to stop”. You can interrupt at any time by pressing
Esc
or by pressing Ctrl+C in the terminal. Claude will release the lock, restore all apps, and return control.
Security mechanisms and the trust boundary
The official documentation specifically emphasizes that, unlike sandboxed Bash tools, Computer Use runs directly in your desktop environment—so the trust boundary is different. Built-in safety guardrails include:
Per-app authorization: Claude can only control the apps explicitly authorized in the current session
Sentinel warnings: Apps granted access to the shell, filesystem, or system settings will be specially marked before authorization
Automatic prompt injection detection: Claude checks the screen content and flags potential prompt-injection attacks
Usage requirements and limitations
Requires subscribing to the Pro or Max plan (Team, Enterprise plans are not applicable)
Requires Claude Code v2.1.85 or above
Currently only supports macOS (Research Preview stage)
Must be used in an interactive session; it does not support non-interactive mode ( -p flag )
Apps such as browsers and trading platforms are limited to “view mode”; the terminal and IDE are “click mode”; other apps are full control
MacRumors notes that, compared with competitors, Claude’s Computer Use takes a more stringent permission-first design, and it is currently limited to Mac, while other solutions have already supported macOS, Windows, and Linux.
For developers, the biggest value of this feature is shortening the loop of “write code → test → fix bugs”—Claude not only writes code, but can also personally watch your screen to validate the results.
Risk warning: Investing in cryptocurrencies carries a high level of risk, and prices may fluctuate drastically. You may lose all of your principal. Please carefully assess the risks.
This article, Claude Code new feature: AI directly controls your Mac screen, from writing code in the terminal to end-to-end click testing, first appeared on Chain News ABMedia.
