Hello! Just wondering whether it is possible to set max depth for the crawl? Previous posts (2023) seems to make use of 'userData' to track the depth. Thank you.

Someone will reply to you shortly. In the meantime, this might help:

Hi, this can be set. You can add a maxdepth parameter in the INPUT_SCHEMA.json file, then retrieve it from the code and implement the specified depth functionality.

@mjh Hey, when using python, you can use

max_crawl_depth

(https://crawlee.dev/python/api/class/BasicCrawlerOptions#max_crawl_depth) . Otherwise, you can track the current depth using

request.userData

. When adding new links in the request handler, just increment the depth by one and use that for each newly enqueued request (https://crawlee.dev/api/core/class/Request)

Got this. thank you