Tutorials

Browser Automation

Browser Automation

Drag and Drop with Image-Based Selectors

The Selector System Hardware-Level Interaction Automation Engines Match Criteria & Filtering

Complete Program

Here's the whole workflow, start to finish. Each piece is broken down and explained below.

using GenerallyPositive;

using GenerallyPositive.Image;

using GenerallyPositive.Browser;

using static GenerallyPositive.Enums;


// a picture of the close button, saved beside the workflow

string imageData = @"close-button.png";


Selector closeButtonSelector = (Selector)GPAL.Selector

.WithCSS(@"#dialog > div > button")

.WithXPath(@"//*[@id=""dialog""]/div/button")

.WithText(@"x")

.MatchText(@"x")

.WithOffsetX(26)

.WithOffsetY(18)

.WithImage(imageData);


Selector mapSelector = (Selector)GPAL.Selector

.WithCSS(@"#map")

.WithXPath(@"//*[@id=""map""]")

.ContainsText(@"OpenStreetMap contributors")

.WithOffsetX(10)

.WithOffsetY(12)

.WithDeltaX(-524)

.WithDeltaY(3);


// image selectors need the GPAL.Image matcher, registered once at startup

GPALImage.Register();


IBrowser browser = GPAL.Browser

.WithBrowserType(BrowserType.Chrome)

.WithLoadImages(true)

.WithBlockPopUps(true)

.WithDriverLocation("C:\drivers")

.WithAutomationEngine(AutomationEngine.PuppeteerPort)

.GoTo("https://example-maps.com/")

.ToGPALObject();


browser

.LeftClick(closeButtonSelector);


browser

.DragAndDrop(mapSelector);


browser.Close(true);

A Selector Can Carry Many Clues at Once

closeButtonSelector stacks CSS, XPath, expected text (both WithText and MatchText), and a reference image (WithImage) all on the same selector. GPAL doesn't require you to pick just one strategy - it can use whichever combination of clues is available to find and confirm the element. WithOffsetX and WithOffsetY then say exactly where, relative to that element, the click should land. Image matching comes from the GPAL.Image package: add it and call GPALImage.Register() once at startup, before the first image selector runs.

Selector closeButtonSelector = (Selector)GPAL.Selector

.WithCSS(@"#dialog > div > button")

.WithXPath(@"//*[@id=""dialog""]/div/button")

.WithText(@"x")

.MatchText(@"x")

.WithOffsetX(26)

.WithOffsetY(18)

.WithImage(imageData);

NOTE

WithImage takes image data (here, captured ahead of time as a base64 string in a variable called imageData) that GPAL can use to visually confirm or locate the element on screen. This is most useful when CSS and XPath alone aren't reliable, such as canvas-based UI or elements that shift position. See The Selector System.

Close the Popup First

Before dragging anything, the workflow left-clicks the popup's close button. This is a normal LeftClick - no different from clicking any other element - it just happens to be the cleanup step that needs to run before the map underneath is usable.

browser

.LeftClick(closeButtonSelector);

DragAndDrop Uses Offsets and Deltas

mapSelector defines a starting point on the map (via WithOffsetX/WithOffsetY, relative to the matched element) and a movement distance (WithDeltaX/WithDeltaY - here, moving left 524 pixels and down 3 pixels). DragAndDrop performs a press-move-release sequence from the starting point across that delta, using the current engine's own drag events: Selenium Actions, CDP mouse events for Puppeteer, or in-page events for OttoMagic.

Selector mapSelector = (Selector)GPAL.Selector

.WithCSS(@"#map")

.WithXPath(@"//*[@id=""map""]")

.WithOffsetX(10)

.WithOffsetY(12)

.WithDeltaX(-524)

.WithDeltaY(3);


browser

.DragAndDrop(mapSelector);

WARNING

DragAndDrop falls back to moving the real mouse cursor only for image-matched or remote-desktop elements. That hardware drag requires the browser window to have focus and be in front. If another window steals focus mid-drag, the drag can land in the wrong place or fail entirely. See Hardware-Level Interaction.

Behind the Scenes: Every Engine, Every Mode

The actual test program in the repo loops over every BrowserType, AutomationEngine, and headless/headful combination, running this same close-then-drag sequence each time. That's purely for regression coverage across engines - your own workflow only needs the single pass shown above. See Automation Engines for what each engine offers.

On this page

Browser Automation
Drag and Drop with Image-Based Selectors
  • Complete Program
  • A Selector Can Carry Many Clues at Once
  • Close the Popup First
  • DragAndDrop Uses Offsets and Deltas
  • Behind the Scenes: Every Engine, Every Mode
    • 💬 Ask GPAL