This is the forum of the developer community of Apify. 

Find answers to pre-existing questions about Crawlee, Apify, and our other open-source projects here.

Apify

I'm seeing a lot of the same exact URL's being ran twice? Any ideas?

Hi. Hm...that is not the default behavior 🤔.
The equality of two requests is determined via comparison of their `uniqueKey` properties. By changing this property value, you could be able to process the same URLs multiple times (if their `uniqueKey` values were unique ofc). Do you explicitly set this property?

See [this](https://crawlee.dev/api/core/class/Request#uniqueKey).