AI Agent Tools: Computer Use

cua32.2k★

Cua gives AI agents a computer of their own: a full macOS or Linux desktop they can click and type through, instead of your screen. The repo bundles Cua Driver (agents operate native apps on macOS, Windows and Linux), Lume (local VMs on Apple Silicon), an SDK and CLI, and Cua Bench for testing agents. Driver, Lume and the SDK are MIT and free. Setup is a curl | sh installer. The Spaces app needs macOS 26 or later, Lume needs Apple Silicon, and Windows is a preview. You bring the machines: your Mac, a spare Mac mini, or VMs in your own AWS, Google Cloud or Modal account. Cua rents nothing, so it isn't a drop-in for E2B. Spaces is free for 2 live sessions and 20 relay hours a month; local-network Spaces don't count. Pro is $18/month, Teams $30/seat/month. Solo: free. Small teams: free on one network or over Tailscale; Teams once you want shared machines with roles. Larger teams: wait, SSO hasn't shipped. The catch: this project deprecates its own floor fast. Five packages are retired (one with known security issues), hosted Cloud Fleets end on October 16, 2026, and Spaces is FSL, not MIT, until each release converts two years out.

opencli31.0k★

OpenCLI turns websites into terminal commands and lets an AI agent drive your logged-in Chrome. Built-in adapters like opencli hackernews top return tables or JSON, and opencli browser commands let an agent navigate, click, fill forms and extract from any page without an adapter. It's free and Apache 2.0. Setup is an npm install plus the Browser Bridge Chrome extension, which talks to a local daemon and drives separate Chrome windows that reuse your site logins. Agents get it through skills (npx skills add jackwener/opencli). The adapter list covers 100+ sites with a heavy lean toward Chinese platforms like Bilibili, Zhihu and Xiaohongshu. The optional OpenCLIApp tray app is closed source. There's no paid tier. You pay only for the LLM tokens your agent spends. Solo and small teams: free, for automating your own accounts. Larger teams: it's per-person tooling, not shared infrastructure; for headless browsers at scale, look at Browserbase. The catch: an agent in your logged-in browser can do anything you can on every site you're signed into. The extension requests debugger, cookie and all-sites access to do it. It also sheds features fast: the CLI hub and desktop-app automation were cut from the main branch this fall.

jev-ultrafast24.4k★

jev-ultrafast is Browser Use's speed demo for a new kind of model. Normal browser agents ask a chat model to write out its next step. This one hands Jev, a decision model from TypeSafe AI that returns typed choices instead of text, a numbered list of page elements and gets back which one to click. A small text model only writes the words that need typing. The README's showcase is a Zurich to London search on Google Flights in about seven seconds. MIT licensed. It is source code, not a product: uv sync, a local Chrome with remote debugging on, and two paid keys. One is a TypeSafe key for Jev, which bills $0.042 per million input tokens with free output and is waitlisted right now. The other is an OpenRouter key for the text model. No releases, version 0.1.0, and the README calls it an MVP. Solo builders curious about decision models: worth an evening once you clear the waitlist. Teams that need browser automation today should use the same company's main browser-use library, or Skyvern. The catch: the benchmark is three repeats of one task on one browser profile, and the README says so. Shadow DOM, iframes, canvas, uploads, and pop-up tabs are out of scope. And the whole thing depends on a closed API you cannot self-host.

browser-harness19.0k★

Browser Harness lets your coding agent (Claude Code, Codex or Cursor) drive your real Chrome, logged in as you: open pages, click, type, read and download. It talks to Chrome over the DevTools Protocol, and when a helper is missing the agent writes one. Free and MIT licensed, from the Browser Use team. Setup is one prompt pasted into your agent, which installs it with uv and has you tick remote debugging in chrome://inspect. Cursor and other MCP clients use the bundled MCP server. It's no longer a tiny script: it ships interaction guides and a growing library of site-specific skills. In Browser Use's own benchmark, no tool beat it on score and it used the fewest tokens. Local use costs nothing. Browser Use Cloud is pay-as-you-go at $0.02 per browser-hour plus $5/GB for residential proxies. Solo and small teams: free on your own Chrome. Larger teams running many browsers: the cloud is cheap. The catch: it gets your whole logged-in browser, and telemetry is on by default. Each run sends the script your agent wrote and up to 20,000 characters of its output to PostHog, which can include page text from your accounts. Run browser-harness telemetry disable first.

OpenBot11.2k★

OpenBot is CopilotKit's template for a company AI assistant you run yourself: a ChatGPT-style app where each agent gets its own container with a browser, files and only the tools you grant. Every action passes one gateway that checks your policy, writes an audit row, then acts. The code is MIT. Setup is Docker and Bun, or one published image with Postgres built in. The docs say 2GB of RAM minimum, 4GB recommended, plus about 1GB per browser running at once. Any AG-UI agent plugs in (LangGraph, Mastra, CrewAI or hand-written), and you bring your own model key. Sign-in ships switched off, so turn it on before anyone else can reach it. Here's the paywall: the server refuses to start without CopilotKit Intelligence, their proprietary service for threads and memory. Free hosted covers 1,000 interactions a month per organization; pay as you go is $49/month with 10,000 included. Solo: free. Small teams: free until 1,000 a month, then $49. Larger teams: an Annual contract, custom priced. The catch: MIT code, rented brain. Running Intelligence on your own servers in production is an Enterprise feature. It's also labeled alpha, about ten weeks old, and CopilotKit sells a finished version, Soar, separately.

skill-recorder5.8k★

Skill Recorder watches you do a task once and turns it into a skill an AI agent can repeat. It's a free, MIT-licensed Microsoft desktop app: hit record, do the work (narrate if you like), and the GitHub Copilot CLI rebuilds it as an intent plus ordered steps, then writes a reusable SKILL.md or a scheduled automation. There's no installer. Releases are source-only: a pasted command fetches a pinned Node runtime and builds the app on your machine, macOS first, with Windows 11 and Ubuntu supported. Recording and Whisper transcription stay local. Clicking Analyze sends window titles, URLs, clipboard previews, terminal commands, screenshots and narration to GitHub's cloud. Skills install directly into Scout, Microsoft's Autopilot agent, or export for Copilot Cowork or any skill-capable agent. Copilot Studio is still marked coming soon. Solo and small teams: free, with a Copilot plan for the analysis (Pro is $10/month). Larger Microsoft shops: the real bill is Microsoft 365 Copilot at $30/user/month paid yearly, plus usage-based Cowork. The catch: nothing has hit main since 0.7.0 on September 21 and it's still a pre-1.0 experiment. The README also warns against recording passwords or tokens, which rules out a lot of real admin work.

iPhone-use5.0k★

iPhone Use lets Codex operate your real iPhone over USB. Describe a task in plain language and the agent opens apps, reads the screen, taps, scrolls and types, while a sidebar shows the phone live. Under the hood it's Appium's WebDriverAgent on the phone, driven by a local MCP server (the plug-in protocol AI assistants use to call tools), with no separate Appium server. Free, MIT. Setup is the hard part. You need a Mac with full Xcode (Command Line Tools won't do), Python, Node.js, Developer Mode on the iPhone, and an Apple account in Xcode to sign the WebDriverAgent runner. A free Personal Team works, but Apple expires those profiles after 7 days, so plan on rebuilding weekly. The setup skill walks Codex through it; Apple login, trust prompts and unlocking stay with you. Solo: free, and it works with Codex, WorkBuddy or Doubao Work. Teams doing app QA: use a proper Appium setup or a real-device cloud like BrowserStack; this is built for one person's phone. The catch: the Codex version sends anonymous usage analytics to the author's PostHog project by default. Phone images and typed text stay local, per the docs, and IPHONE_USE_ANALYTICS=0 turns it off.

phone-harness4.9k★

phone-harness hands an AI agent your phone. Claude Code, Codex or any shell-capable agent gets eyes (on-screen text with tap coordinates) and hands (taps, swipes, typing) on a real iPhone or Android, with no jailbreak and nothing installed on the phone. MIT licensed and free. iPhone runs through the Mac's iPhone Mirroring window with Apple's Vision OCR, so it needs macOS Sequoia, and the window stays in front unless you opt into an experimental background mode. Android goes over adb from macOS, Linux or Windows and reads the accessibility tree. iPhone Mirroring still isn't offered in the EU, so EU users get Android or cloud. No spare phone? Phone Harness Cloud rents hosted iPhones and Androids at 5 cents a minute per phone, with $5 of promo credit to start; dedicated phones are custom-priced. Solo tinkerers: free on your own device. Teams doing real mobile testing: Appium, Maestro and BrowserStack remain the serious answers. The catch: you're giving an LLM your logged-in phone, and telemetry is on by default. It sends your script, helper-call arguments (including text you type) and up to 20,000 characters of output to PostHog. Run phone-harness config set telemetry false first.

esp32-iphone-agent4.9k★

ESP32 iPhone Agent (now branded Handsy AI) lets an AI model drive a stock iPhone through a $7 microcontroller board. The board is both an AirPlay screen, so the model sees the phone, and a USB mouse and keyboard, so it can tap, swipe and type. The agent app on the phone works with any OpenAI-compatible vision model. Free, open source, no jailbreak. The board is a Seeed XIAO ESP32S3, $7.49 at Seeed. Flash it with arduino-cli or handsy.tech's browser installer. The app isn't on the App Store, so you build it in Xcode on a Mac. A free Apple ID can sign it but must re-sign every 7 days, and the one-tap hotspot join needs a paid developer account. Solo tinkerers: free, a fun weekend build. Teams doing app QA: mobile-mcp or Midscene skip the hardware, at the cost of a Mac and code signing. The catch: it's slow. The public firmware mirrors the screen over Bluetooth at 480x1040 and 5 fps at most, so steps take seconds. It types ASCII only. And the browser installer flashes firmware built from a commit that isn't in the public repo, so build from source to know what's typing into your phone.

chrome-cdp-skill3.3k★

This skill connects your agent to your live Chrome via the Chrome DevTools Protocol (CDP). Your agent can read pages, click buttons, fill forms, and navigate, in the browser you're already using. The difference from tools like Playwright is that this connects to existing tabs. Your agent can interact with pages where you're already authenticated, see what you see, and do what you'd do manually. MIT licensed, JavaScript. The catch: giving an AI agent access to your live browser session with all your logged-in accounts is a real security consideration. The agent can see everything you can see, including sensitive data in open tabs. There's no permission model beyond 'full access.' And CDP connections can be fragile; Chrome updates can break the protocol.

AutoCLI3.0k★

AutoCLI, formerly opencli-rs, pulls data from websites with one terminal command: Hacker News top stories, a Reddit search, trending Bilibili videos, returned as a table, JSON, CSV or Markdown. It's a Rust rewrite of OpenCLI that ships as a single small binary covering 55 sites, Apache 2.0 and free. Public sites work right after install. For sites that need a login, you sideload a Chrome extension and AutoCLI reuses your logged-in browser session, so it acts on those sites as you. It also wraps gh, docker and kubectl for AI agents. Generating adapters for new sites goes through autocli.ai and needs its token. There's no paid tier. Solo tinkerers and agent builders: free. Teams: think hard before pointing something that scrapes as the logged-in user at company accounts, and expect site-specific commands to break whenever a site changes its layout. The catch: it looks abandoned. The last commit and release were April 20, 2026, community pull requests sit unmerged, users report broken Twitter and Google commands, and autocli.ai returned a 502 error when we checked. The upstream OpenCLI it was rewritten from shipped v2.0.0 in October 2026. Use that instead.

chromex1.1k★

Chromex is a Chrome side-panel extension that connects your browser to OpenAI's Codex through a local native messaging bridge. Summarize pages, work across tabs and screenshots, edit images, transcribe voice, and run browser-control workflows with visible in-page indicators. MIT licensed. Setup is heavier than a typical extension: clone the repo, npm install && npm run build, run install-native-host.mjs, then load the unpacked extension at chrome://extensions. The architecture (Chrome extension to native host to local bridge to codex app-server) keeps your API key local; raw keys aren't stored in extension storage. Pick this if you live in Chrome, already run Codex, and want one assistant that sees the page you're on. Solo: free, you pay only for Codex tokens. Small teams: same. Large teams or non-Codex shops: skip; this is built around Codex specifically. The catch: Codex-only. Switch to Claude or Gemini for your CLI agent and Chromex doesn't follow. The native bridge is only as polished as the project, which is small and early. For a more mature Chrome AI assistant, Sider and Monica have more features and broader model support.