@Maxwell for bulk WordPress discovery at that volume, brute-force crawling isn't really the play, a fingerprint-and-filter approach works much better. WordPress sites are identifiable cheaply: wp-json on a HEAD request, wp-content showing up in the HTML, or the generator meta tag if the site hasn't stripped it. Feed candidate domains from something that already indexes broadly, Common Crawl's domain index or certificate transparency logs if you want closer to real-time, then fingerprint-check each one instead of rendering full pages. At 1M a day you're mostly bottlenecked on the candidate-domain source, not the WordPress check itself, that part's fast.