Tools/onehumanbeing/esp32-iphone-agent

esp32-iphone-agent

Let an AI model use a real iPhone with a $7 ESP32: AirPlay eyes, USB HID hands, any OpenAI-compatible vision model. No Mac, no developer account, no jailbreak.

3.6k ★+940/wkemergingPythonMIT + GPL-3.0 (firmware)new this week

The Lens

By Erik Loyd, SaaS CEO and former COO/CFO of an AWS Premier Partner.

Updated Oct 2026

ESP32 iPhone Agent (now branded Handsy AI) lets an AI model drive a stock iPhone through a $7 microcontroller board. The board is both an AirPlay screen, so the model sees the phone, and a USB mouse and keyboard, so it can tap, swipe and type. The agent app on the phone works with any OpenAI-compatible vision model. Free, open source, no jailbreak.

The board is a Seeed XIAO ESP32S3, $7.49 at Seeed. Flash it with arduino-cli or handsy.tech's browser installer. The app isn't on the App Store, so you build it in Xcode on a Mac. A free Apple ID can sign it but must re-sign every 7 days, and the one-tap hotspot join needs a paid developer account.

Solo tinkerers: free, a fun weekend build. Teams doing app QA: mobile-mcp or Midscene skip the hardware, at the cost of a Mac and code signing.

The catch: it's slow. The public firmware mirrors the screen over Bluetooth at 480x1040 and 5 fps at most, so steps take seconds. It types ASCII only. And the browser installer flashes firmware built from a commit that isn't in the public repo, so build from source to know what's typing into your phone.

Free vs Self-Hosted vs Paid

fully free

Free: The iOS app and firmware are MIT, except the AirPlay receiver code in hardware/hid_bridge/src/airplay/ (ported from UxPlay), which is GPL-3.0, so the flashed firmware binary is GPL-3.0. No paid tier, no account. The app you build from the main branch has no analytics and no developer server (the log uploader ships with no destination). The project site's privacy policy describes a coming store build with opt-in PostHog usage stats and an optional free "Handsy line" model run by the developer, which keeps each step's screenshot and your task; neither is in the main branch yet.

Self-hosted: Hardware is one Seeed Studio XIAO ESP32S3 (plain version, 8 MB PSRAM; not Sense, not Plus), $7.49 on Seeed's store; any ESP32-S3 with native USB and PSRAM should work. The iPhone needs iOS 18 or later and cellular data; Lightning iPhones need a Lightning to USB Camera Adapter. Flash the board with arduino-cli (ESP32 core 3.3.x, NimBLE-Arduino 2.5.x) or the browser installer at handsy.tech (firmware v0.2.1, built from a commit not published on GitHub). Build the app in Xcode on a Mac and sign it yourself: a free Apple ID works with a 7-day re-sign. The one-tap hotspot join needs the Hotspot Configuration capability (a paid Apple developer account); without it you type the hotspot password by hand.

Paid: No paid tier. The running cost is the vision model: any OpenAI-compatible API that accepts images, billed by whichever provider you point it at, or your own server (vLLM, SGLang, llama.cpp, Ollama). Every step sends a screenshot.

Free code and a $7.49 board. The ongoing cost is your vision-model bill, which grows with every screenshot the agent sends per step, or nothing if you run the model on your own hardware.

What to do by team size

Solo
free; a $7.49 board, a Mac with Xcode to build the app once, and a vision model endpoint
Small team
free, but one board drives one phone at a few seconds per step; fine for personal automation, not a test lab
Medium team
use WebDriverAgent-based tooling or a real-device cloud for QA instead
Large team
not a fit; device-farm tooling instead
Self-hosting ops:moderate

Get tools like this every Wednesday

One featured tool, three on the radar. No fluff.

Score
67/100 · B
Adoption17/30
Maintenance25/25
Community5/20
License10/15
Analysis10/10

A low score is not a verdict on quality. Young and niche tools start low by design. How we calculate scores

License: MIT + GPL-3.0 (firmware)

App code is MIT; the flashed firmware links GPL-3.0 UxPlay code, so the binary is GPL-3.0.

Commercial use: ✓ Yes

About

Owner
Henry Yuan (User)
Stars
3,582
Forks
472

Explore Further

More tools in the directory