Hello, Is there a way to get the load time of a site from crawlee in headless mode? I'm using PlaywrightCrawler. Thanks!

Someone will reply to you shortly. In the meantime, this might help:

Hi @BOBPG : Do you have something like this in your mind?

typescript

const crawler = new PlaywrightCrawler({
    // ...
    preNavigationHooks: [async ({ request }, gotoOptions) => {
        requestStartTimes.set(request.id!, Date.now());
        gotoOptions!.waitUntil = 'load';
    }],
    requestHandler: async ({ request, log }) => {
        const loadingTime = Date.now() - requestStartTimes.get(request.id!)!;
        log.info(`Loading ${request.url} took ${loadingTime / 1000}s`);
    },
});

Someting like this looks great, i'm going to try this, thanks!