I finished the crawler with around 1.7M, and got around 100k failed requests. Is there a way to retry just the failed requests ?

Someone will reply to you shortly. In the meantime, this might help:

hey, that's not currently supported, I would recommend creating a dataset/kv store for failed requests and push to it from failed request handler

Retrying it would necessitate writing a new scraper since the crawler consists of multiple routes handlers.... I thought I could just run the scraper again but increasing the retry count.

@Vi just advanced to level 3! Thanks for your contributions! 🎉

you can use failed request handler to handle all failed request in one place https://crawlee.dev/api/playwright-crawler/interface/PlaywrightCrawlerOptions#failedRequestHandler