Tutorials

Browser Automation

Browser Automation

Testing Stealth Against Anti-Bot Detection Sites

Automation Engines Conditional Logic Browser Profiles: Signed-In vs Temporary The Event System Selector Matching

Complete Program

Here's a trimmed-down version of the workflow, start to finish. Each piece is broken down and explained below.

using System.Collections.Generic;

using GenerallyPositive;

using GenerallyPositive.Browser;

using static GenerallyPositive.Enums;


GPAL

.CallIfNotFound(GenericCallIfNotFound)

.WithAutomationEngine(AutomationEngine.OttoMagic)

.WithPublishToConsole()

.WithAutoUpdateWebDriver();


List<string> urls = new List<string>

{

"https://bot-detection.example.com/score",

"https://fingerprint.example.com/check",

"https://example.com/cookie-wall"

};


IBrowser browser = GPAL.Browser

.WithBrowserType(BrowserType.Chrome)

.WithUseStealth(StealthType.DarkMode | StealthType.GoogleReferrer | StealthType.PatchDriver | StealthType.CDP)

.WithLoadImages(true)

.WithWindowSize(new System.Drawing.Rectangle(10, 10, 2000, 2000))

.WithDriverLocation("C:\drivers")

.WithAutomationEngine(AutomationEngine.OttoMagic)

.WithProfileDataDirectory("C:\Profiles\Stealth")

.ToGPALObject();


foreach (string url in urls)

{

browser.GoTo(url);

browser.WaitFor(5_000);


if (true == UrlHelper.AreEquivalent(url, "https://example.com/cookie-wall"))

{

browser.WaitFor(7_000);

browser.LeftClick("#cookie-accept-all");

}


for (int page = 0; page < 10; page++)

{

string fileName = $"pix\page_{page}.PNG";

GPALFile screenCapFile = GPAL.File.WithFileName(fileName).ToGPALObject();

GPAL.CaptureScreen(browser).SaveTo(screenCapFile);


_ = browser.PageDown;


if (0 < page && true == browser.IsEndOfPage())

break;

}

}


browser.Close(true);


CallIfStatus GenericCallIfNotFound(IBrowser browser, List<IGPALElement> foundElements, List<IGPALElement> matchedElements, Selector selector, bool matchedAll)

{

string message = $"Unable to locate Selector [{selector.Name}] [{selector.SelectorPaths[0].SelectorPath}]";

GPAL.PublishSimpleEvent(GPALEventType.FAILURE, message, browser, GPALObjectType.Browser);

return CallIfStatus.Handled;

}

A Global CallIfNotFound Handler

GPAL.CallIfNotFound registers a workflow-wide handler that fires whenever a selector fails to match, on top of any per-selector CallIfNotFound. Here it publishes a FAILURE event with the selector's name and first path, then returns CallIfStatus.Handled so the workflow continues instead of throwing.

GPAL

.CallIfNotFound(GenericCallIfNotFound)

.WithAutomationEngine(AutomationEngine.OttoMagic)

.WithPublishToConsole()

.WithAutoUpdateWebDriver();


CallIfStatus GenericCallIfNotFound(IBrowser browser, List<IGPALElement> foundElements, List<IGPALElement> matchedElements, Selector selector, bool matchedAll)

{

string message = $"Unable to locate Selector [{selector.Name}] [{selector.SelectorPaths[0].SelectorPath}]";

GPAL.PublishSimpleEvent(GPALEventType.FAILURE, message, browser, GPALObjectType.Browser);

return CallIfStatus.Handled;

}

NOTE

The real test program keeps a List of selectors that are expected to be missing on some sites, and checks selector.Name against it before logging - so genuinely optional selectors don't spam the error log.

Stealth, Window Size, and a Persistent Profile

WithUseStealth combines several flags - dark mode, a Google referrer header, a patched chromedriver binary, and CDP leak protection - to make the browser look less like an automated session. WithProfileDataDirectory points at a real Chrome profile directory, so cookies, history, and fingerprint-relevant state persist between runs instead of starting from a blank slate every time.

IBrowser browser = GPAL.Browser

.WithBrowserType(BrowserType.Chrome)

.WithUseStealth(StealthType.DarkMode | StealthType.GoogleReferrer | StealthType.PatchDriver | StealthType.CDP)

.WithLoadImages(true)

.WithWindowSize(new System.Drawing.Rectangle(10, 10, 2000, 2000))

.WithDriverLocation("C:\drivers")

.WithAutomationEngine(AutomationEngine.OttoMagic)

.WithProfileDataDirectory("C:\Profiles\Stealth")

.ToGPALObject();

NOTE

StealthType is a flags enum - combine as many as the target site needs with bitwise OR. See Browser Profiles: Signed-In vs Temporary for more on persistent profile directories.

Per-Site Conditional Handling

Different anti-bot pages need different one-off handling: some need extra wait time before a fingerprint check finishes, others need a cookie banner dismissed before screenshots make sense. UrlHelper.AreEquivalent compares the current URL against a known target regardless of trailing slashes or minor formatting differences, so each branch only runs for the site it applies to.

if (true == UrlHelper.AreEquivalent(url, "https://example.com/cookie-wall"))

{

browser.WaitFor(7_000);

browser.LeftClick("#cookie-accept-all");

}

Capture a Scroll-Through Sequence

The inner loop takes up to 10 screenshots, scrolling down with browser.PageDown between each. GPAL.CaptureScreen(browser).SaveTo writes each frame to a GPALFile-backed PNG, and browser.IsEndOfPage() stops the loop early once scrolling has reached the bottom.

for (int page = 0; page < 10; page++)

{

string fileName = $"pix\page_{page}.PNG";

GPALFile screenCapFile = GPAL.File.WithFileName(fileName).ToGPALObject();

GPAL.CaptureScreen(browser).SaveTo(screenCapFile);


_ = browser.PageDown;


if (0 < page && true == browser.IsEndOfPage())

break;

}

WARNING

Programmatic scrolling is itself a signal some anti-bot systems watch for. If a site's detection score changes after scrolling, that's the page reacting to PageDown, not a GPAL bug.

Nested BrowserType / AutomationEngine Loops

The real test program wraps all of this in nested foreach loops over the browsers and engines checked on its form, using continue to skip the pairs that cannot run: Puppeteer drives Chromium only, so Firefox with any of the four Puppeteer engines is skipped. This is a common pattern for regression-testing a workflow across every supported configuration without duplicating the workflow code.

// browsers and engines hold the BrowserType and AutomationEngine values checked on the form

foreach (BrowserType bt in browsers)

{

foreach (AutomationEngine ae in engines)

{

// puppeteer drives chromium only

if (BrowserType.FireFox == bt && (AutomationEngine.PuppeteerPort == ae || AutomationEngine.PuppeteerPortHW == ae ||

AutomationEngine.PuppeteerPipe == ae || AutomationEngine.PuppeteerPipeHW == ae))

continue;


// build browser and run the workflow body here

}

}

NOTE

A continue for each pair you want to skip keeps the loop body focused on the combinations you actually care about, and makes it obvious at a glance which engines and browser types are in scope for a given test run. See Automation Engines and Conditional Logic.

On this page

Browser Automation
Testing Stealth Against Anti-Bot Detection Sites
  • Complete Program
  • A Global CallIfNotFound Handler
  • Stealth, Window Size, and a Persistent Profile
  • Per-Site Conditional Handling
  • Capture a Scroll-Through Sequence
  • Nested BrowserType / AutomationEngine Loops
    • 💬 Ask GPAL