Tutorials

REST and APIs

REST and APIs

Seeing What a Page Asks For with CaptureCalls

Seeing What a Page Asks For with .CaptureCalls Templates: Repeating the Call a Page Made Browser Automation Basics Automation Engines

Complete Program

The whole workflow. It opens a page, records what that page asked the network for, and prints the calls worth calling yourself. Each piece is broken down below.

using System;

using System.Collections.Generic;

using GenerallyPositive;

using GenerallyPositive.Browser;

using static GenerallyPositive.Enums;


GPAL.WithPublishToConsole();


IBrowser browser = GPAL.Browser

.WithBrowserType(BrowserType.Chrome)

.WithAutomationEngine(AutomationEngine.PuppeteerPort)

.WithDriverLocation(@"C:drivers")

.ToGPALObject();


browser

.WithCallFilter("/api/") // only the calls with this in the url are kept

.CaptureCalls() // recording starts here, so say it before you navigate

.GoTo("https://www.example.com/rooms")

.CaptureCalls(out List<GPALCall> calls);


foreach (GPALCall call in calls)

GPAL.PublishSimpleEvent(GPALEventType.NOTICE,

$"[{call.Method}] [{call.Status}] [{call.ResourceType}] [{call.Url}]");


browser.Close(true);

Record Before You Navigate

CaptureCalls turns recording on, and from that point every request the browser makes is kept: method, url, resource type, status, headers and body. Nothing before the call is recorded, so it belongs ahead of the GoTo that triggers the traffic you want. Pass false to stop.

browser

.CaptureCalls()

.GoTo("https://www.example.com/rooms");

NOTE

Puppeteer reports requests over CDP, OttoMagic watches with the extension's webRequest listener, and Selenium records them in the page through the driver. All three can capture. A browser with none of those available says so once rather than quietly recording nothing.

Narrow It With WithCallFilter

A modern page asks for three hundred things and three of them matter. WithCallFilter keeps only the calls whose url contains the fragment you give it, so the list you read back is the endpoint rather than the fonts, images and analytics around it. Say it before capturing.

browser

.WithCallFilter("/availability-api/")

.CaptureCalls()

.GoTo("https://www.example.com/rooms");

WARNING

Nothing is missed by declaring the filter late, only unrecorded. Traffic that went past before capturing started was never kept, and no filter brings it back.

Read the Calls Back

The out overload hands back everything recorded so far, as a list of GPALCall. Url, Method, ResourceType, Status, Initiator and PostData are all there, and Headers holds what the page's own code added, which is the part a request built by hand always leaves out. Status is what the page itself got, so it separates an endpoint worth calling from one the page is failing at too.

browser.CaptureCalls(out List<GPALCall> calls);


foreach (GPALCall call in calls)

{

GPAL.PublishSimpleEvent(GPALEventType.NOTICE, $"[{call.Method}] [{call.Url}]");


foreach (KeyValuePair<string, string> header in call.Headers)

GPAL.PublishSimpleEvent(GPALEventType.NOTICE, $" [{header.Key}] [{header.Value}]");

}

NOTE

Calling CaptureCalls again keeps recording on for any tab opened since, and hands back everything since the browser started. A long-running spy can poll it on a timer without losing anything.

From a Call to a Request You Can Issue

Reading the list tells you what the endpoint is. CaptureCallTemplate goes one step further and hands the call back as a GPALRequest, ready to issue: the path, the method, the body and the headers the page's own code added. That is how a workflow carries a value it cannot know, a token minted per session or a signed parameter, because the template comes from the run that is happening rather than from something pasted in last week.

browser

.WithCallFilter("/availability-api/multiple-price-from")

.CaptureCallTemplate(out GPALRequest priceFrom);


if (null == priceFrom)

GPAL.PublishSimpleEvent(GPALEventType.CAUTION, "The page never made that call, so there is nothing to repeat");

WARNING

CaptureCallTemplate blocks until a matching call arrives or it runs out of time, and the last match wins, since a page that calls an endpoint more than once has settled by the last one. When nothing matched it hands back null rather than throwing, so the workflow decides what that means. Check for null before using it.

Where It Goes Next

A template on its own is a description. Give it to .Fetch and the page issues it again, carrying the session it already earned. That is the next tutorial in this section.

browser

.WithTokensFrom(nights)

.Fetch(priceFrom)

.SaveTo(ref json);

On this page

REST and APIs
Seeing What a Page Asks For with CaptureCalls
  • Complete Program
  • Record Before You Navigate
  • Narrow It With WithCallFilter
  • Read the Calls Back
  • From a Call to a Request You Can Issue
  • Where It Goes Next
    • 💬 Ask GPAL