Tutorials

Tutorials

A workflow that pulls a random, highly-rated book from one site, then opens a second site in a new tab to search for it and scrape the price. Covers persistent selectors for cookie/login popups, NewTab and CloseTab, and reacting to search results with CallIfFound and InElement.

Complete Program

The shape of the workflow: configure GPAL and a persistent selector for a cookie popup, load a list page, pull rating info and titles into grids, then for each book that clears a rating bar, open a second site in a new tab, search for it, and scrape the result.

using GenerallyPositive;

using GenerallyPositive.Browser;

using static GenerallyPositive.Enums;

GPAL

.WithExceptionHandler(ExceptionEventHandler)

.WithDriverLocation(@"c:drivers")

.WithPublishToConsole();

IGPALGrid<string> outputGrid = GPAL.Grid.ToGPALObject();

Selector cookiesPopup = GPAL.Selector

.WithCSS("#onetrust-close-btn-container > button")

.CallIfFound(CloseCookiesPopup)

.WithSelectorName("Persistent: Cookies Popup")

.ToGPALObject();

IBrowser browser = GPAL.Browser

.WithBrowserType(BrowserType.Chrome)

.WithPersistentSelector(cookiesPopup)

.WithWaitOnDocumentReady(60_000)

.ToGPALObject();

Selector ratingsSelector = GPAL.Selector.WithXPath("//span[contains(@class, 'rating-count')]").WithSelectorName("Ratings").ToGPALObject();

Selector titleSelector = GPAL.Selector.WithXPath("//a[contains(@class, 'item-title')]").UseAttribute("text").WithSelectorName("Title").ToGPALObject();

browser

.GoTo("https://www.example-books.com/random")

.WithSelector(ratingsSelector)

.WithSelector(titleSelector)

.WithAllThatMatch(-1)

.GetGrid(out IGPALGrid<string> resultsGrid);

string bookTitle = resultsGrid[0][1];

Selector searchBar = GPAL.Selector.WithCSS("input.search-input").WithSelectorName("Search Bar").ToGPALObject();

Selector searchButton = GPAL.Selector.WithCSS("button.search-submit").WithSelectorName("Search Button").ToGPALObject();

Selector searchResults = GPAL.Selector.WithCSS("div.product-tile").WithSelectorName("Search Results").ToGPALObject();

browser.NewTab("https://www.example-retailer.com/")

.WithSelector(searchBar)

.FillInFrom(bookTitle)

.WithSelector(searchButton)

.LeftClick();

browser

.WithSelector(searchResults)

.WaitFor(5_000)

.CallIfFound(SelectFirstResult)

.WithAllThatMatch(-1)

.StartWorkflow();

browser.CloseTab();

browser.Close(true);

Persistent Selectors Survive Page Navigation

WithPersistentSelector attaches a selector to the browser itself, not to a single unit of work. GPAL checks for it on every page and runs its CallIfFound handler whenever it appears - perfect for cookie banners or login popups that can pop up at any time. Once handled, the callback removes the selector and unregisters itself so it does not keep firing.

Selector cookiesPopup = GPAL.Selector

.WithCSS("#onetrust-close-btn-container > button")

.CallIfFound(CloseCookiesPopup)

.WithSelectorName("Persistent: Cookies Popup")

.ToGPALObject();

IBrowser browser = GPAL.Browser

.WithBrowserType(BrowserType.Chrome)

.WithPersistentSelector(cookiesPopup)

.WithWaitOnDocumentReady(60_000)

.ToGPALObject();

public static CallIfStatus CloseCookiesPopup(IBrowser browser, List<GPALElement> foundElements, List<GPALElement> matchedElements, Selector selector, bool matchedAll)

{

foreach (GPALElement element in foundElements)

element.Click();

selector.Remove();

browser.RemoveCallIfHandlerEverywhere(CloseCookiesPopup);

return CallIfStatus.Handled;

}

TIP

Calling Remove() on the selector and RemoveCallIfHandlerEverywhere() on the browser stops GPAL from re-checking for a popup that has already been dismissed. Skipping this means the same handler fires repeatedly for the rest of the run.

Pull a Candidate List Into a Grid

WithAllThatMatch(-1) means 'every match, no limit'. GetGrid pulls every matched element from every WithSelector call into one in-memory grid - one column per selector, one row per matched element - so you can reason about results in plain C# afterward.

browser

.GoTo("https://www.example-books.com/random")

.WithSelector(ratingsSelector)

.WithSelector(titleSelector)

.WithAllThatMatch(-1)

.GetGrid(out IGPALGrid<string> resultsGrid);

string bookTitle = resultsGrid[0][1];

Open a Second Site in a New Tab

NewTab opens a fresh tab and navigates it, without losing the original tab's state. From there the workflow is the same shape as any other: fill in a search box, click search, and wait for results.

browser.NewTab("https://www.example-retailer.com/")

.WithSelector(searchBar)

.FillInFrom(bookTitle)

.WithSelector(searchButton)

.LeftClick();

WARNING

If you open extra tabs with NewTab, call CloseTab() to return to the original tab before the final Close(true). Closing the driver while a non-original tab is active can leave the workflow in an inconsistent state on the next run.

React to Search Results with CallIfFound and InElement

CallIfFound runs a callback only when WithSelector actually matched something - here, SelectFirstResult clicks the first product tile. Inside that callback, InElement scopes further selectors to just that element, so a title or price lookup only searches within the clicked result, not the whole page.

browser

.WithSelector(searchResults)

.WaitFor(5_000)

.CallIfFound(SelectFirstResult)

.WithAllThatMatch(-1)

.StartWorkflow();

private static CallIfStatus SelectFirstResult(IBrowser browser, List<GPALElement> foundElements, List<GPALElement> matchedElements, Selector selector, bool matchedAll)

{

foundElements[0].Click();

Selector priceSelector = GPAL.CssSelector("#pdp-cur-price").WithSelectorName("Price").ToGPALObject();

browser

.WithSelector(priceSelector)

.GetGrid(out IGPALGrid<string> priceGrid);

if (0 < priceGrid?.Count())

outputGrid.AddRow(priceGrid[0]);

return CallIfStatus.Handled;

}

On this page

Tutorials
Cross-Site Workflow: Find a Book, Then Search for It Elsewhere
  • Complete Program
  • Persistent Selectors Survive Page Navigation
  • Pull a Candidate List Into a Grid
  • Open a Second Site in a New Tab
  • React to Search Results with CallIfFound and InElement