Concepts

Featured

Featured

Continuing as a RESTClient: Taking the Session Out of the Browser

What Gets Carried

The client is created against the origin the browser is currently on, so it is ready to be given an endpoint rather than made to restate what the browser already knows. The cookies are read for that origin through the same storage action any workflow uses, so host-only and domain cookies are treated the way the browser treats them. The user agent and accept-language are asked of the running browser rather than taken from what it was configured with, because a site can bind a session to the agent it saw. A credential the browser holds is carried too, which matters for basic authentication, where nothing ever becomes a cookie and the header has to be presented on every request. An endpoint that is already a whole url is used as it stands, so a call to another host on the same session is possible.

browser

.GoTo("https://www.example.com/")

.ContinueAsRESTClient(out IRESTClient client);


string body = client

.WithEndpoint("/api/search")

.WithHttpMethod("GET")

.Execute(false);


// the client keeps working after this

browser.Close(true);

When It Is the Right Route

Wherever a site's wall is the cookie jar, this is the faster and simpler tool: no browser round trip, no page needed, and it can run in parallel once the session exists. It is also the route that works cross-origin, since an HTTP client has no origin and no CORS to obey, which is exactly where .Fetch cannot help. Whether a given site accepts it is not predictable from the protection vendor. The same vendor answers one site's endpoint and refuses another's, so the useful thing is to try both routes against the same session and read the two answers rather than reason about which should work.

// the same call, both ways, against one session

browser.Fetch(request).SaveTo(ref fromThePage);


browser.ContinueAsRESTClient(out IRESTClient client);


string fromOutside = client

.WithEndpoint(request.Path)

.WithParameters(request.Body)

.WithHttpMethod("POST")

.Execute(false);

NOTE

Both through means the wall was cookies and the fast route is available. In-page through and browserless refused means the wall is the transport. Neither means the address, or a page that never settled.

What It Cannot Carry

It cannot carry the transport. An HTTP client produces its own TLS handshake, its own HTTP/2 settings and its own header order, and a protection layer that fingerprints the connection rejects it before any application code sees the request. Correct cookies do not change that. It also cannot carry what the page holds in memory, so a value the site's JavaScript computes per request has to be captured and handed over deliberately. Run the in-page route first, because it can never put the session at risk, and the browserless one second: a replay the edge refuses can get the session invalidated for the browser that earned it, so an in-page call that worked a moment ago starts failing too. When that happens, navigate again rather than retrying.

WARNING

Measured against an Akamai-protected site: after a refused browserless call, the in-page fetch on the same browser began failing as well. Order matters, and a fresh navigation is the remedy rather than a retry.

On this page

Featured
Continuing as a RESTClient: Taking the Session Out of the Browser
  • What Gets Carried
  • When It Is the Right Route
  • What It Cannot Carry
    • 💬 Ask GPAL