macOS platform guide for the agent-desktop CLI. Use for desktop automation on macOS: granting Accessibility and Screen Recording permission (TCC), what headed mode does on macOS (physical-first click, type and scroll), surfaces (menu, menubar, sheet, popover, alert, dock, menu-bar-extras), menu bar apps and system panels with no window, Notification Center, Chromium and Electron apps including launch --cdp, and fixing PERM_DENIED, empty accessibility trees, ACTION_FAILED and APP_UNRESPONSIVE.
Repository statistics
Indexed skills
8
Skill groups
1
GitHub stars
25,777
Forks
1,932
Skills
8 totalOperate desktop applications on macOS, Windows and Linux through native accessibility trees, using the agent-desktop CLI. Use when an AI agent must see or act on a desktop GUI: click buttons, fill forms, read UI state, navigate menus, toggle checkboxes, scroll, drag, type text, press shortcuts, take screenshots, manage windows, use the clipboard, launch or close apps, read notifications. Triggers on: "click button", "fill form", "open app", "read UI", "computer use", "operate desktop", "accessibility tree", "snapshot app", "type into field", "navigate menu", "toggle checkbox", "take screenshot", "desktop automation", "agent-desktop", or any desktop GUI task on macOS, Windows or Linux.
Linux platform guide for the agent-desktop CLI. Lists per command group what works on X11 and on Wayland, how to check the AT-SPI accessibility bus, the first-contact gotchas (app names, Chromium and Electron flags, headed focus, blocked key combos), and where to find lifecycle, screenshot, clipboard, tray, notification and troubleshooting details. Triggers on: "linux desktop automation", "AT-SPI tree", "automate GNOME app", "agent-desktop on Linux", "accessibility bus not available", "empty supported_surfaces", "wayland screenshot consent", or any Linux GUI interaction task.
Windows platform guide for the agent-desktop CLI. Lists per command group what works on Windows and what refuses, the first-contact gotchas (PowerShell ref quoting, minimized windows, headed-only input), and where to find shell surfaces, notifications, the cursor overlay, UIPI elevation, Chromium/Electron settle behavior and troubleshooting. Triggers on: "windows desktop automation", "UIA tree", "automate Windows app", "agent-desktop on Windows", "elevated window input blocked", "E_ACCESSDENIED", or any Windows GUI interaction task.
C-ABI bindings over agent-desktop's PlatformAdapter. Consumers (Python ctypes, Swift, Node ffi-napi, Go cgo, C++, Ruby fiddle) link libagent_desktop_ffi.{dylib,so,dll} and call `ad_*` functions directly instead of spawning the CLI binary per call. The canonical observe-act workflow is: ad_init → ad_adapter_create[_with_session] → ad_snapshot → read qualified refs (@s8f3k2p9:e5) → ad_execute_by_ref → ad_free_string → ad_adapter_destroy.
Create or update architecture and flow diagrams for agent-desktop's docs site in apps/docs. Use when a docs page needs a diagram of components, data flow, or a command pipeline, or when an existing diagram in apps/docs/public/desktop/diagrams changes.
Write, review, and edit agent-desktop documentation, including the docs site in apps/docs served at agent-browser.dev/desktop, READMEs, the agent skills in skills/, changelog entries, commit messages, PR descriptions, and issues. Use for getting-started guides, command references, conceptual explanations, integration guides, troubleshooting, and code examples.
Drive a desktop application from one plain-language goal, or resolve one intent at a time, without ever reading the accessibility tree. Use when automating a desktop app and you want to keep a 150-element JSON tree out of your context. Hand `run.mjs` a whole goal and it observes, decides and acts until the goal is met; hand `act.mjs` a single step when you want to keep the plan yourself. Covers which element, which operation, when to look deeper, and when to stop. Needs a clone of the agent-desktop repository and a TYPESAFE_API_KEY.
Add to your README
The badge links readers to this page. It shows the skilld mark and no counts, and it follows the reader's light or dark GitHub theme.
<a href="https://skilld.dev/gh/vercel-labs/agent-desktop">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://skilld.dev/b/vercel-labs/agent-desktop?theme=dark">
<source media="(prefers-color-scheme: light)" srcset="https://skilld.dev/b/vercel-labs/agent-desktop?theme=light">
<img alt="Skill repository on skilld.dev" src="https://skilld.dev/b/vercel-labs/agent-desktop?theme=light">
</picture>
</a>