Concepts

Automation Targets

Automation Targets

Page Readiness: Document Ready vs Network Idle

WithWaitOnDocumentReady and WithWaitOnIdleConnection both make GPAL wait before continuing after a navigation, but they measure 'ready' very differently. And for pages that load data asynchronously, only one of them tells the truth.

WithWaitOnDocumentReady: the Obvious Default

WithWaitOnDocumentReady(timeoutInMs) waits for the browser's own page-load signal -- the same moment a traditional page fires its load event. For a simple server-rendered page, that's a good proxy for 'ready': by the time the document is loaded, the content is there. Switching between WithWaitOnDocumentReady and not using it at all is usually a no-brainer -- it's the setting most people reach for first, and the name says exactly what it does.

GPAL.Browser

.WithBrowserType(BrowserType.Chrome)

.WithWaitOnDocumentReady(10_000)

.GoTo("https://example.com");

WithWaitOnIdleConnection: a Better Signal for Async Pages

Many modern pages fire their load event almost immediately and then make a wave of background requests. API calls that fill in the actual content you care about. On those pages, document-ready fires too early: the DOM exists, but the data you want to scrape hasn't arrived yet. WithWaitOnIdleConnection(true) instead waits until the browser's network activity drops to (and stays at) a small number of connections. WithNetworkIdleMaxConnections, default 0. For a minimum duration. WithNetworkIdleTimeoutMs, default 500ms. In other words, it waits until the page has stopped fetching things, which is a much closer match to 'the content is actually here' for async-heavy pages.

GPAL.Browser

.WithBrowserType(BrowserType.Chrome)

.WithWaitOnIdleConnection(true)

.WithNetworkIdleMaxConnections(0) // wait until 0 connections are active...

.WithNetworkIdleTimeoutMs(1_000) // ...for at least 1 second

.GoTo("https://example.com/dashboard");

TIP

WithNetworkIdlePruneMs (default 10 seconds) controls how long GPAL retains connection history when deciding whether the network is idle. A page that polls an endpoint every few seconds may never report 0 active connections for long. In that case, raise WithNetworkIdleMaxConnections slightly rather than waiting indefinitely.

Which One Should You Use?

Start with WithWaitOnDocumentReady -- it's simple, fast, and correct for the majority of pages. Reach for WithWaitOnIdleConnection specifically when a workflow's selectors aren't being found even though the page 'looks loaded' in a screenshot -- that's the signature of a page whose real content arrives after an async fetch. The two aren't mutually exclusive: you can set both, and GPAL will satisfy whichever condition takes longer.

WARNING

Document-ready is a familiar concept from general web development, so toggling WithWaitOnDocumentReady on or off feels obvious. Network-idle waiting is GPAL-specific and far less discoverable. If a workflow intermittently misses content on a data-heavy page, this section is the one to come back to.

On this page

Automation Targets
Page Readiness: Document Ready vs Network Idle
  • WithWaitOnDocumentReady: the Obvious Default
  • WithWaitOnIdleConnection: a Better Signal for Async Pages
  • Which One Should You Use?