Concepts

Automation Targets

Automation Targets

Hardware-Level Interaction

WithHardware switches an element's interaction from programmatic DOM/UI-Automation calls to real OS-level mouse and keyboard events. The same input a person generates. And it works the same way for both browsers and desktop applications.

What WithHardware Changes

By default, GPAL interacts with elements through the automation engine's normal API. Selenium's element.Click(), Puppeteer's CDP input events, or UI Automation's invoke patterns for desktop controls. WithHardware switches that element to real OS-level input instead: GPAL moves the actual mouse cursor and generates real click and keypress events at the OS level, exactly as if a person were at the keyboard. For text inputs under hardware emulation, focusing the control means physically clicking it first; otherwise GPAL just hovers over it.

// Normal interaction (Selenium/Puppeteer/UIA call directly into the element)

browser

.LeftClick("#normal-btn");

// Hardware interaction - real OS mouse movement and click

browser

.WithSelector("#stubborn-btn").WithHardware

.LeftClick();

When Programmatic Interaction Isn't Enough

Some elements don't respond correctly to programmatic events -- custom canvas-drawn controls, drag-and-drop widgets, or sites that specifically check for synthetic input as a sign of automation. WithHardware bypasses all of that by generating the same low-level input a real user produces, at the cost of needing the target element to actually be visible and positioned on screen for the mouse to reach it. After a hardware click, GPAL normally leaves the cursor right where it landed. The one exception is when the current unit of work also uses image selectors: GPAL then parks the mouse at a randomized edge or corner of the browser window first, so a lingering hover state (tooltips, dropdowns, sticky menus) can't obscure the next image match. Workflows without image selectors never see this extra move -- one less mechanical, repeatable pattern in the input stream.

TIP

WithHardware can be set globally on GPAL, inline on the workflow right after WithSelector (overriding only for that step), or baked into a reusable Selector definition. The most specific level wins. See 'Configuration Hierarchy' in Core Concepts for how these layers combine.

Works for Applications Too

Because GPAL.Application shares the same selector and interaction model as GPAL.Browser, WithHardware applies identically to desktop application controls. This is often the more common case for applications. Many native Windows controls only respond fully to real input events, so hardware emulation is the default expectation rather than the exception when automating legacy desktop software.

WARNING

Because WithHardware moves the real mouse and sends real OS input, it will interfere with anything else happening on the machine at the time. Including a user's own mouse and keyboard. Workflows that rely on it are best run on a dedicated machine or in a session the user isn't actively using.

On this page

Automation Targets
Hardware-Level Interaction
  • What WithHardware Changes
  • When Programmatic Interaction Isn't Enough
  • Works for Applications Too