Tutorials

Browser Automation

Browser Automation

Persistent Selectors: Handling Nags and Popups Conditional Logic Selectors Unit of Work GPALGrid: Working with Tabular Data

Complete Program

The shape of the workflow: configure GPAL and a persistent selector for a cookie popup, load a list page, pull rating info and titles into grids, then for each book that clears a rating bar, open a second site in a new tab, search for it, and scrape the result.

using System;

using System.Collections.Generic;

using System.Diagnostics;

using GenerallyPositive;

using GenerallyPositive.Browser;

using static GenerallyPositive.Enums;


GPAL

.WithExceptionHandler(ExceptionEventHandler)

.WithDriverLocation("c:\drivers")

.WithPublishToConsole();


IGPALGrid<string> outputGrid = GPAL.Grid.ToGPALObject();


Selector cookiesPopup = GPAL.Selector

.WithCSS("#onetrust-close-btn-container > button")

.CallIfFound(CloseCookiesPopup)

.WithSelectorName("Persistent: Cookies Popup")

.ToGPALObject();


IBrowser browser = GPAL.Browser

.WithBrowserType(BrowserType.Chrome)

.WithPersistentSelector(cookiesPopup)

.WithWaitOnDocumentReady(60_000)

.ToGPALObject();


Selector ratingsSelector = GPAL.Selector.WithXPath("//span[contains(@class, 'rating-count')]").WithSelectorName("Ratings").ToGPALObject();

Selector titleSelector = GPAL.Selector.WithXPath("//a[contains(@class, 'item-title')]").UseAttribute("text").WithSelectorName("Title").ToGPALObject();


IGPALGrid<string> resultsGrid = GPAL.Grid.ToGPALObject();

browser

.GoTo("https://www.example-books.com/random")

.WithSelector(ratingsSelector)

.WithSelector(titleSelector)

.WithAllThatMatch()

.GetGrid(ref resultsGrid);


string bookTitle = resultsGrid[0][1];


Selector searchBar = GPAL.Selector.WithCSS("input.search-input").WithSelectorName("Search Bar").ToGPALObject();

Selector searchButton = GPAL.Selector.WithCSS("button.search-submit").WithSelectorName("Search Button").ToGPALObject();

Selector searchResults = GPAL.Selector.WithCSS("div.product-tile").WithSelectorName("Search Results").ToGPALObject();


browser.NewTab("https://www.example-retailer.com/")

.WithSelector(searchBar)

.FillInFrom(bookTitle)

.WithSelector(searchButton)

.LeftClick();


IGPALGrid<string> priceGrid = GPAL.Grid.ToGPALObject();

browser

.WithSelector(searchResults)

.WaitFor(5_000)

.CallIfFound(SelectFirstResult)

.WithAllThatMatch()

.StartWorkflow();


browser.CloseTab();

browser.Close(true);


CallIfStatus CloseCookiesPopup(IBrowser browser, List<IGPALElement> foundElements, List<IGPALElement> matchedElements, Selector selector, bool matchedAll)

{

foreach (IGPALElement element in foundElements)

element.Click();


selector.Remove();

browser.RemoveCallIfHandlerEverywhere(CloseCookiesPopup);

return CallIfStatus.Handled;

}


CallIfStatus SelectFirstResult(IBrowser browser, List<IGPALElement> foundElements, List<IGPALElement> matchedElements, Selector selector, bool matchedAll)

{

foundElements[0].Click();


Selector priceSelector = GPAL.CssSelector("#pdp-cur-price").WithSelectorName("Price").ToGPALObject();


browser

.WithSelector(priceSelector)

.GetGrid(ref priceGrid);


if (0 < priceGrid?.Count())

outputGrid.AddRow(priceGrid[0]);


return CallIfStatus.Handled;

}


void ExceptionEventHandler(object sender, EventArgs e)

{

Debug.WriteLine(((GPAL.GPALEventArgs)e).Message);

Debug.WriteLine(((GPAL.GPALEventArgs)e).ExceptionRaised?.Message);

}

Persistent Selectors Survive Page Navigation

WithPersistentSelector attaches a selector to the browser itself, not to a single unit of work. GPAL checks for it at the start of every action and runs its CallIfFound handler whenever it is on the page - perfect for cookie banners or login popups that can pop up at any time. Once handled, the callback removes the selector and unregisters itself so it does not keep firing.

Selector cookiesPopup = GPAL.Selector

.WithCSS("#onetrust-close-btn-container > button")

.CallIfFound(CloseCookiesPopup)

.WithSelectorName("Persistent: Cookies Popup")

.ToGPALObject();


IBrowser browser = GPAL.Browser

.WithBrowserType(BrowserType.Chrome)

.WithPersistentSelector(cookiesPopup)

.WithWaitOnDocumentReady(60_000)

.ToGPALObject();


CallIfStatus CloseCookiesPopup(IBrowser browser, List<IGPALElement> foundElements, List<IGPALElement> matchedElements, Selector selector, bool matchedAll)

{

foreach (IGPALElement element in foundElements)

element.Click();


selector.Remove();

browser.RemoveCallIfHandlerEverywhere(CloseCookiesPopup);

return CallIfStatus.Handled;

}

NOTE

Calling Remove() on the selector and RemoveCallIfHandlerEverywhere() on the browser stops GPAL from re-checking for a popup that has already been dismissed. Skipping this means the same handler fires repeatedly for the rest of the run.

Pull a Candidate List Into a Grid

WithAllThatMatch() with no argument means 'every match, no limit'; a number caps it. GetGrid pulls every matched element from every WithSelector call into one in-memory grid - one column per selector, one row per matched element - so you can reason about results in plain C# afterward.

IGPALGrid<string> resultsGrid = GPAL.Grid.ToGPALObject();

browser

.GoTo("https://www.example-books.com/random")

.WithSelector(ratingsSelector)

.WithSelector(titleSelector)

.WithAllThatMatch()

.GetGrid(ref resultsGrid);


string bookTitle = resultsGrid[0][1];

Open a Second Site in a New Tab

NewTab opens a fresh tab and navigates it, without losing the original tab's state. From there the workflow is the same shape as any other: fill in a search box, click search, and wait for results.

browser.NewTab("https://www.example-retailer.com/")

.WithSelector(searchBar)

.FillInFrom(bookTitle)

.WithSelector(searchButton)

.LeftClick();

WARNING

If you open extra tabs with NewTab, call CloseTab() to return to the original tab before the final Close(true). Closing the driver while a non-original tab is active can leave the workflow in an inconsistent state on the next run.

React to Search Results with CallIfFound

CallIfFound runs a callback only when WithSelector actually matched something - here, SelectFirstResult clicks the first product tile. Inside that callback, a fresh chain reads the price from the product page the click opened and adds it to outputGrid.

IGPALGrid<string> priceGrid = GPAL.Grid.ToGPALObject();

browser

.WithSelector(searchResults)

.WaitFor(5_000)

.CallIfFound(SelectFirstResult)

.WithAllThatMatch()

.StartWorkflow();


CallIfStatus SelectFirstResult(IBrowser browser, List<IGPALElement> foundElements, List<IGPALElement> matchedElements, Selector selector, bool matchedAll)

{

foundElements[0].Click();


Selector priceSelector = GPAL.CssSelector("#pdp-cur-price").WithSelectorName("Price").ToGPALObject();


browser

.WithSelector(priceSelector)

.GetGrid(ref priceGrid);


if (0 < priceGrid?.Count())

outputGrid.AddRow(priceGrid[0]);


return CallIfStatus.Handled;

}

On this page

Browser Automation
Cross-Site Workflow: Find a Book, Then Search for It Elsewhere
  • Complete Program
  • Persistent Selectors Survive Page Navigation
  • Pull a Candidate List Into a Grid
  • Open a Second Site in a New Tab
  • React to Search Results with CallIfFound
    • 💬 Ask GPAL