CheerioCrawler headerGenerator help
# crawlee-js
n
Hello ! I kept reading the docs but couldn't find a clear information about this. When we use Puppeteer or Playwright we can tweak in browserPool the fingerprintGenerator. For Cheerio we have the headerGenerator from got, how we can adjust it inside the CheerioCrawler ?
h
Someone will reply to you shortly. In the meantime, this might help:
l
Here's an example on how to work with headerGeneratorOptions using the BasicCrawler. I would assume it works in the same way for the CheerioCrawler. https://crawlee.dev/docs/next/guides/got-scraping#useheadergenerator
f
Hi! You could also attempt to add the following option to `CheerioCrawler`:
Copy code
preNavigationHooks: [
            async (crawlingContext, opts: OptionsInit) => {
                opts.headers = {
                    ...opts.headers,
                    // your headers
                };
            }
        ]