Understudy is a local-first desktop agent runtime that can operate GUI apps, browsers, terminals, and messaging tools in a single session. The creator built it because most AI agents today only work within one surface - either a browser or a chat interface - while real work spans multiple applications. Understudy's key innovation is teach-by-demonstration, where you show it a task once and it learns the intent rather than just recording coordinates.

In the demo, the agent learns to Google search, download an image, remove the background in Pixelmator Pro, export it, and send it via Telegram. Then it can repeat the process for a different subject without brittle macro recording. The system stores intent steps, route options, and GUI hints as fallback rather than hardcoded coordinates. When faster execution routes exist, it can prefer those over repeating every GUI step.

The current implementation works on macOS only with Layers 1-2 complete and Layers 3-4 partially implemented. You can install it with `npm install -g @understudy-ai/understudy` and run the wizard to get started. The GitHub repository includes the full source code and examples.