Library

Selector

Selector

Click Offset and Image Matching

WithOffsetX and WithOffsetY shift where a click lands relative to the top-left corner of the matched element - useful when a click must land in a specific spot within a larger element. WithDeltaX and WithDeltaY define the drop offset for drag-and-drop operations. WithImage uses template matching from the GPAL.Image package, registered once with GPALImage.Register(), to find an element by visual appearance rather than by DOM properties - valuable for canvas elements, legacy applications, or any UI that does not expose automation properties. WithOffsetX and WithOffsetY shift the click within a matched image the same way they do within a matched element. A matched image is then treated like any other selector. GPAL asks what is under it, the page element in a browser or the control in an application, walking up from a picture or a label to the link, button or field it belongs to, and that element answers for the match: fill-in, text, values, attributes, select and upload act on it. A click, a hover or a drag still goes to the matched point, since the element can be much larger than the picture and its middle is not where the picture is. In a browser the engine clicks that point itself unless the workflow asks for hardware, through a hardware engine, WithHardware or WithSimulateMouse; in an application a point is always clicked with the mouse, which is the only way Windows offers. On OttoMagic the browser lookup needs Allow user scripts turned on for the extension. In a browser the picture is searched for in a capture the browser takes of its own page, not in a capture of the screen, so image selectors work headless and on a hidden desktop, on every engine. In an application the search is made on the screen. This makes an image a good last path on a selector, for sites that rename their elements to stop automation. Every other path, CSS, XPath, text, value, name, class, placeholder and href, reads the markup, so a rewrite of the page can break them all at once. A picture only stops matching when the look changes, and a site rarely changes both at once.

NOTE

Each attempt takes a capture and compares it, so an image path is slower than CSS or XPath. Declare it last on the selector. The paths that read the markup run first, and the image runs only when they all find nothing, which is the moment the page has changed under them.

Examples

GPAL Fluent: High-level fluent C# API

//WithImage accepts a file path (string), a GPALFile, or a System.Drawing.Image object. The template image should be a clean crop of the target element at the same screen resolution and zoom level.

// Click 10px right and 5px down from element top-left

var offsetClick = GPAL.Selector

.WithCSS(".chart-area")

.WithOffsetX(10)

.WithOffsetY(5);


// Find and click a button by its visual appearance. Once, at startup:

GPALImage.Register();


var imgBtn = GPAL.Selector

.WithImage("submit-button.png")

.WithOffsetX(5)

.WithOffsetY(3);

On this page

Selector
Click Offset and Image Matching
  • Examples
    • 💬 Ask GPAL