Shell surfaces, notifications and cursor overlay
Open this file to raise shell surfaces, read or drive notifications, use the cursor overlay, or handle hosted (UWP) window identity and menu detection.
Shell Surfaces
open-system-surface --surface <kind> raises a shell surface and returns the
w-<hwnd> identity of the window it presents. Windows kinds: start-menu,
taskbar, system-tray, system-tray-overflow, action-center. Read the
surface with snapshot --surface <kind>. snapshot --window-id refuses that
handle with WINDOW_NOT_FOUND, because the window inventory excludes shell
windows.
- The command takes the foreground. Strict headless refuses it with
POLICY_DENIEDbefore anything is raised. Pass global--headed. An already-present surface is returned without being raised again. snapshot --surface <kind>needs no--appand works headless. A closed surface returnsWINDOW_NOT_FOUNDwith a suggestion that namesopen-system-surface.quick-settingsreturnsINVALID_ARGSand namesaction-center, where the quick actions live.start-menuis whatever the Win key raises. Before Windows 11 that is a search overlay whose root holds aSearchTextBox, not a tile grid. Drive it as a search surface.- Tray items use the generic commands. There are no Windows-specific tray
commands.
snapshot --surface system-trayreturns the promoted notification-area items; zero items is a correct answer on a machine with no tray icons.snapshot --surface taskbargives refs to the notification area's traybuttons.snapshot --surface system-tray-overflowgives refs to the overflow items: raise the flyout first withopen-system-surface --surface system-tray-overflow, or a click fails the occlusion check. A click on asystem-trayitem reportsdelivered_unverifiedwithretry: unsafe, because a synthesized click cannot confirm what the owning app did with it. A click on an overflow item is untested.
Notifications
The notification commands read and drive the Action Center through UI
Automation. The WinRT UserNotificationListener is never used, so its
per-machine consent setting cannot fail a call. The reader keys on landmark
AutomationIds: MainListView when notifications are present, an empty-state
shape when none are, and a top-level ClearAllButton.
- The center must be open for a read.
list-notificationsreads an already-open center headless. When the center is closed, strict headless returnsPOLICY_DENIEDbefore raising it, and--headedraises it and restores the desktop afterwards.dismiss-notification,dismiss-all-notificationsandnotification-actionalways need--headed, andwait --notificationrefuses on its first poll under the same rule. - An empty center lists zero entries. A center that shows neither the
notification list nor the empty-state landmarks returns
PLATFORM_NOT_SUPPORTEDwith aplatform_detailthat names the missing landmark. An unrecognized tree is an error, never an empty answer. - The index is not the identity. Every mutation re-reads the center and
compares the entry at the index with
--expected-appand--expected-title. A mismatch isNOTIFICATION_NOT_FOUND. An action name the entry does not offer isACTION_NOT_SUPPORTED. - A dismiss that does nothing fails. The shell can accept an entry's
DismissButtonwithout acting on it. When the target is the only entry, the verified clear-all control runs instead. Otherwise the call reportsACTION_FAILED(delivered_unverified).dismiss-all-notificationsis judged against the entries captured before the clear, so entries that arrive while it runs are not failures. wait --notificationopens and closes the center per poll, as on macOS. Each poll adopts an already-open center and restores the entry state after. Measured cost: 1243.5 ms per poll at the minimum, 1254.2 ms at the median. The wait also sleeps up to 500 ms between polls, so size--timeoutfor both. A notification that appears and is dismissed inside one interval can be missed. A toast joins the center only while the center is open, so toasts posted between polls never land. To stage arrivals, hold the center open yourself; the polls adopt it without closing it.- This output is sensitive. The notification-area surface shows the names
of installed background agents, including security and remote-access
products, and
list-notificationsreturns titles and bodies verbatim. Nothing is redacted.
Cursor Overlay
cursor-overlay enable renders on Windows: a detached renderer process paints a
presentation-only cursor in a click-through, topmost, non-activating window.
$env:AGENT_DESKTOP_HOME = "$env:TEMP\ad-scratch"
$start = agent-desktop session start --cursor | ConvertFrom-Json
$env:AGENT_DESKTOP_SESSION = $start.data.session_id
agent-desktop cursor-overlay enable --label "Opening menu" --accent "#FF3B7B"
agent-desktop cursor-overlay disable- Start the session before you snapshot. The overlay needs a session, and a
snapshot taken outside it lands in the global namespace, where a
session-scoped action cannot see it (
SNAPSHOT_NOT_FOUND). Runsession start,cursor-overlay enable,snapshot, then act. data.renderedis the renderer's acknowledgement.enablereturnstrueonly after a renderer connects to the session's control pipe and acknowledges the control. It returnsfalsewhen no renderer could be reached, spawned or heard from in time.disablecarries norenderedfield.session start --cursoralso enables the overlay and echoes onlycursor_overlay.- Headed actions skip the travel animation. A
--headedaction moves the real cursor, so the overlay does not animate travel or the click for it. An enabled overlay still paints its resting cursor. - It costs about 355 ms per action. A headless
clicktook 427 ms without the overlay and 782 ms with it. Enabling costs about 50 ms once. A hundred overlaid clicks add about 35 seconds. - It starts at the centre of the primary monitor's work area, not over the
app you drive. The first frame lands shortly after
enablereturns, so poll for the pixels before you take a screenshot. - It draws above the taskbar and other topmost shell chrome.
- It ignores the "Show animations in Windows" setting. Windows reports
animations off by default on a stock Windows Server host, so the explicit
enablecounts as the opt-in. macOS honours Reduce Motion instead. - The label is a black pill with white Segoe UI text, sized to the text.
--filland--rimcolour only the cursor, not the label. - The label shows only when there is something to say.
enablewithout--labelgreets once, and with--imageit does not greet. After that, only an action that carries a description draws the label. Eachenableand each action replaces the label text. - It fades out after 6 s with no instruction and comes straight back on the next command.
- The caret comes from the Text pattern. After text entry, the agent's caret sits where the field's Text pattern places the insertion point, in a window of its own, and goes when the cursor sets off for somewhere else. A field without the Text pattern keeps the element outline.
- It draws in physical pixels, the space element bounds are reported in, so it is not scaled again for the monitor's DPI. Multi-monitor placement is unit-tested only and not verified on a live multi-monitor desktop.
- A harness job object takes the overlay with it. A job with
JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE, which CI runners and some agent harnesses use, ends the detached renderer when the job closes. A vanished overlay there is not a renderer defect, and actions are unaffected. - Its control pipe trusts any process that runs as you. See "The Cursor
Overlay's Control Pipe" in
references/permissions-and-elevation.md. - Teardown.
cursor-overlay disableends the renderer for its session, andsession endcloses it too. When a session ends another way (a crash,session gc), the renderer reclaims itself within 3000 ms.
Hosted (UWP) Window Identity
An app hosted in ApplicationFrameHost is reported through its frame:
focused_window and list-windows give the frame's handle as the window id,
which every window operation targets, while app and pid name the hosted
app, read from its Windows.UI.Core.CoreWindow. A suspended hosted app drops
its CoreWindow, so until it resumes the entry reads as its frame host, and
identities checked against the hosted pid fail closed.
Menu Detection Coverage
The menu surface detector covers classic Win32 menus, WPF context menus and
menu-bar dropdowns, and Chromium/Electron DOM context menus drawn inside the
app's own window. WinUI3 and MSIX hosts are not evaluated. For an app outside
these families, read "no menu is open" as "not detected", not as proof that the
menu is closed.