Library

Browser

Browser

Data Retrieval

GetGrid retrieves the matched selector data into an IGPALGrid<string> for further processing. It takes the grid by ref: pass a grid and GetGrid ADDS its rows into it (it never replaces your reference), so reusing one grid across several GetGrid or multi-page calls accumulates a combined dataset; pass null to get a fresh grid back. Duplicate-row detection is opt-in - call .WithDedupeData(true) on the grid to have GPAL skip duplicate rows. Duplicates are common in the wild and are not tied to infinite scroll: eBay, for example, returns duplicate search results on an ordinary paged run. It is off by default because it fingerprints every element to detect duplicates, which slows GetGrid down (noticeably on slower engines like Selenium); for large result sets, deduping the raw extracted data with another tool may be a better fit than deduping at the element level. GetPageSource extracts the full HTML of the current page. GetSiteMap retrieves the sitemap XML and GetSiteMapUrls returns the individual URLs as a list. GetHydratedData extracts Next.js hydration data embedded in the page. GetLLMDigest produces a cleaned, structured summary of the page content optimized for use as LLM input. The Save variants (including SaveSiteMapUrls) write these outputs directly to files. WithDefaultErrorPlaceholder sets the text written into a grid cell GetGrid could not read, so a missing value is visible rather than blank.

NOTE

GetGrid's parameter is ref, not out. If you pass an existing grid it appends the results into that grid and leaves your reference intact - that is how you accumulate rows from several searches or pages into one master grid. Passing null hands you back GPAL's internal grid instead. Duplicate-row detection is opt-in via .WithDedupeData(true): use it when a site serves duplicate results and you want GPAL to remove them. Duplicates are not unique to infinite scroll - we saw eBay return duplicate results on a normal, non-infinite-scroll run. It is off by default and it slows GetGrid because it hashes each element's attributes to fingerprint every row; for big result sets, deduping the raw extracted data with another tool is often a better fit than deduping at the element level.

Examples

GPAL Fluent: High-level fluent C# API

//GetGrid adds rows into the grid you pass by ref (one row per matched element, columns per selector) - reuse a grid to accumulate across calls, or pass null for a fresh one. .WithDedupeData(true) makes GPAL drop duplicate rows (sites like eBay serve them); it is off by default because fingerprinting each element slows GetGrid.

// Extract a table into a grid (pass by ref)

var grid = GPAL.Grid.ToGPALObject();

GPAL.Browser

.GoTo("https://example.com/data")

.WithSelector(".data-row")

.WithAllThatMatch(1000)

.GetGrid(ref grid);


// Accumulate the rows from several searches into ONE grid

var master = GPAL.Grid.ToGPALObject();

foreach (var term in searchTerms)

GPAL.Browser

.GoTo($"https://example.com/search?q={term}")

.WithSelector(".result-row")

.GetGrid(ref master); // each call appends into master


// Opt in to duplicate-row detection - some sites (e.g. eBay) return duplicate results even on a normal run

var deduped = GPAL.Grid.ToGPALObject().WithDedupeData(true);

GPAL.Browser

.GoTo("https://www.ebay.com/sch/i.html?_nkw=widget")

.WithSelector(".s-item")

.WithAllThatMatch(200)

.GetGrid(ref deduped); // GPAL skips duplicate rows - slower, it fingerprints each element


// Get page HTML source

string html;

GPAL.Browser

.GoTo("https://example.com")

.GetPageSource(out html);


// Get all URLs from a sitemap

List<string> urls;

GPAL.Browser

.GoTo("https://example.com")

.GetSiteMapUrls(out urls);

On this page

Browser
Data Retrieval
  • Examples
    • 💬 Ask GPAL