Tutorials

Getting Started

Getting Started

Simple Scraper: Search, Paginate, and Save

Selectors Selector Matching Unit of Work Event System Automation Engines

Complete Program

Here's the whole workflow, start to finish. Each piece is broken down and explained below.

using GenerallyPositive;

using GenerallyPositive.Browser;

using static GenerallyPositive.Enums;


GPAL

.WithPublishToConsole()

.WithAutoUpdateWebDriver()

.WithTypingDelay(50)

.WithSimulateMouseMovement(true)

.WithDriverLocation(@"c:drivers");


IGPALGrid<string> retGrid;


Selector nextPageButton = "//a[contains(@class, 'pagination__next icon-link')]";

nextPageButton.WithHardware.WithSelectorName("NextPageButton");


Selector imageColumn = "//img[contains(@class, 's-card__image')]";

imageColumn.WithSelectorName("Image");


Selector descriptionColumn = "//span[contains(@class, 'su-styled-text primary default')]";

descriptionColumn.WithSelectorName("Description");


Selector linkColumn = "//a[contains(@class, 's-card__link')]";

linkColumn.WithSelectorName("Link");


Selector priceColumn = "//span[contains(@class, 'su-styled-text primary bold large-1 s-card__price')]";

priceColumn.WithSelectorName("Price");


Selector searchButton = "#gh-search-btn";

searchButton.WithSelectorName("SearchButton").MatchText("Search");


Selector inputSearchSelector = "#gh-ac";

inputSearchSelector.WithSelectorName("SearchInput");


IBrowser browser = GPAL.Browser

.WithBrowserType(BrowserType.Chrome)

.WithUseAutomationEngine(AutomationEngine.PuppeteerPort)

.WithWaitOnDocumentReady(30_000)

.GoTo("https://www.ebay.com/")

.ToGPALObject();


browser

.WaitFor(2_000)

.WithSelector(inputSearchSelector)

.FillInFrom("Dog Treats")

.LeftClick(searchButton)

.WithSelector(descriptionColumn)

.WithSelector(priceColumn)

.WithSelector(linkColumn)

.WithSelector(imageColumn)

.WithAllThatMatch(10)

.CallIfFound(CallIfFound)

.CallIfNotFound(CallIfNotFound)

.WithNextPageButton(nextPageButton)

.WithPages(3)

.GetGrid(out retGrid)

.SaveToTabbedText(@"c: emp esults.txt")

.Close(true);

Configure GPAL & Define Your Selectors

Every workflow starts with a few global settings - here we turn on console logging, auto-update the web driver, and add a human-like typing delay. Then we define each Selector once, give it a friendly name, and (for the search button) require its text to match "Search". Selectors are just values - define them once, reuse them anywhere in the workflow.

GPAL

.WithPublishToConsole()

.WithAutoUpdateWebDriver()

.WithTypingDelay(50)

.WithSimulateMouseMovement(true)

.WithDriverLocation(@"c:drivers");


Selector searchButton = "#gh-search-btn";

searchButton.WithSelectorName("SearchButton").MatchText("Search");


Selector inputSearchSelector = "#gh-ac";

inputSearchSelector.WithSelectorName("SearchInput");

NOTE

WithSelectorName isn't just cosmetic - it's the name GPAL uses when it logs information and exception events for that selector, which makes troubleshooting much easier. See Selectors and Selector Matching for the full picture.

Launch the Browser & Navigate

GPAL.Browser is a fluent factory: pick a browser type, pick an automation engine, tell it to wait until the page is ready, and go to a URL. ToGPALObject() returns the live IBrowser you'll drive for the rest of the workflow.

IBrowser browser = GPAL.Browser

.WithBrowserType(BrowserType.Chrome)

.WithUseAutomationEngine(AutomationEngine.PuppeteerPort)

.WithWaitOnDocumentReady(30_000)

.GoTo("https://www.ebay.com/")

.ToGPALObject();

NOTE

Swapping AutomationEngine.PuppeteerPort for Selenium, OttoMagic, or any of the other 7 engines doesn't change anything below this line. See Automation Engines.

One Chain, One Workflow

This is the heart of GPAL. WithSelector adds elements to the current unit of work - here, the description, price, link, and image columns for up to 10 results per page. WithAllThatMatch tells GPAL how many matching elements to use. No loops: GPAL applies each action across every matched element for you.

browser

.WaitFor(2_000)

.WithSelector(inputSearchSelector)

.FillInFrom("Dog Treats")

.LeftClick(searchButton)

.WithSelector(descriptionColumn)

.WithSelector(priceColumn)

.WithSelector(linkColumn)

.WithSelector(imageColumn)

.WithAllThatMatch(10)

React, Paginate, and Save

CallIfFound and CallIfNotFound let you react to whether a selector matched anything - here we just print element info or show a message box. WithNextPageButton plus WithPages handles pagination automatically. GetGrid pulls everything into an in-memory grid, and SaveToTabbedText writes it straight to a file.

.CallIfFound(CallIfFound)

.CallIfNotFound(CallIfNotFound)

.WithNextPageButton(nextPageButton)

.WithPages(3)

.GetGrid(out retGrid)

.SaveToTabbedText(@"c: emp esults.txt")

.Close(true);

WARNING

Passing true kills the running web driver process. Use true when the workflow is completely done; use false if another workflow in the same run still needs that driver.

On this page

Getting Started
Simple Scraper: Search, Paginate, and Save
  • Complete Program
  • Configure GPAL & Define Your Selectors
  • Launch the Browser & Navigate
  • One Chain, One Workflow
  • React, Paginate, and Save
    • 💬 Ask GPAL