Tutorials

Browser Automation

Browser Automation

Image Matching and Selectors Inside Iframes

Selectors Selector Matching Hardware-Level Interaction Automation Engines

Complete Program

Here's the whole workflow, start to finish. Each piece is broken down and explained below.

using GenerallyPositive;

using GenerallyPositive.Image;

using GenerallyPositive.Browser;

using static GenerallyPositive.Enums;


Selector imageSelector = (Selector)GPAL.Selector

.WithImage((GPALFile)"C:\images\ebay.saved.plus.png")

.WithOffsetX(20)

.WithOffsetY(10);


Selector outerIframeSelector = (Selector)GPAL.Selector

.WithCSS(@"#Multiple > iframe")

.WithSelectorName("outer iframe");


Selector innerIframeSelector = (Selector)GPAL.Selector

.WithCSS(@"html > body > section > div > div > iframe")

.WithSelectorName("inner iframe");


Selector iframeInputSelector = (Selector)GPAL.Selector

.WithCSS(@"html > body > section > div > div > div > input")

.WithOffsetX(112)

.WithOffsetY(5)

.WithSelectorName("iframe input");


// image selectors need the GPAL.Image matcher, registered once at startup

GPALImage.Register();


IBrowser browser = GPAL.Browser

.WithBrowserType(BrowserType.Chrome)

.WithAutomationEngine(AutomationEngine.PuppeteerPort)

.WithDriverLocation("c:\drivers")

.GoTo("https://example.com/frames")

.LeftClick(outerIframeSelector)

.InFrame(outerIframeSelector)

.InFrame(innerIframeSelector)

.LeftClick(iframeInputSelector)

.FillInFrom("Hello World")

.ToGPALObject();


browser

.GoTo("https://www.example.com/")

.WithSelector(imageSelector)

.Hover()

.Close(true);

Matching an Element by Image

WithImage points a Selector at a saved screenshot of the element you want to find, instead of (or in addition to) CSS/XPath. GPAL matches that image against a capture the browser takes of its own page, so it works headless and on a hidden desktop, and the match resolves to the page element under it: text and values come from that element, while a click or hover goes to the matched point. WithOffsetX and WithOffsetY shift the click/hover point relative to the matched image's origin - useful when the actual clickable target is a few pixels away from the most reliable visual anchor. Image matching comes from the GPAL.Image package: add it and call GPALImage.Register() once at startup, before the first image selector runs.

Selector imageSelector = (Selector)GPAL.Selector

.WithImage((GPALFile)"C:\images\ebay.saved.plus.png")

.WithOffsetX(20)

.WithOffsetY(10);

NOTE

GPAL.File.WithFileName(...).ToGPALObject() wraps the screenshot path as a GPALFile, the same value object used for uploads and downloads elsewhere in GPAL. See GPALFile: Files as Objects.

Defining Iframe Selectors

Iframes are just selectors that happen to point at an