How to configure PlaywrightCrawler or PuppeteerCra...
# crawlee-js
w
Hello, I have no success in setting up either of the crawlers with Browserless. What is the proper Typescript way to do that with Browserless? Any docs? Thank you! ⁨⁨```ts const crawler = new PuppeteerCrawler({ launchContext: { useChrome: true, launcher: puppeteer, launchOptions: { browserWSEndpoint: wsEndpoint, => not even connected with this option args: [ '--no-sandbox', '--disable-setuid-sandbox', '--disable-dev-shm-usage' ] } }, ```⁩⁩ ⁨⁨```ts const crawler = new PlaywrightCrawler({ launchContext: { launchOptions: { browserWSEndpoint: wsEndpoint, => not even has this property }, }, ```⁩⁩
a
message has been deleted
l
If you want to use a crawler that just handles HTTP requests you should use the Cheerio Crawler https://crawlee.dev/js/docs/guides/cheerio-crawler-guide
h
I am self hosting crawlee and running playwrightCrawler and I add 100-150 requests in queue and and maxComcurrency is 12 and the crawlee don’t finish all the requests that I added. It is never started the crawler for 3 url. I try different urls different websites but it is skipping or whatever but it is no started or no pick 2-4 request and from 100-150
m
Hey @Hamid Jutt, could you please provide more details? I don't really understand what you are saying. > I am self hosting crawlee and running playwrightCrawler So you are running the Actor locally, not on the Apify platform? > I add 100-150 requests in queue and and maxComcurrency is 12 and the crawlee don’t finish all the requests that I added What is the reason it doesn't finish the requests? Does it fail/crash? Or does it just ignore some of the requests (could be the case if you run it without purging the RequestQueue first, or you have a maxRequests limitation set up). Also if your requests share url (
#
part is ignored) the requests are processed as single unless you set a specific
uniqueKey
for each request.
h
Yes, I am running the actor locally. There is not any crash, it is just ignored. I also set uniqueKey,
a
@Hamid Jutt just advanced to level 1! Thanks for your contributions! 🎉
a
message has been deleted