No matter which params i change my cpu is always a...
# crawlee-js
m
Hi 🙂 , Here my code
Copy code
const configuration = {
  purgeOnStart: false,
  availableMemoryRatio: 0.75,
  maxMemoryOverloadedRatio: 0.75,
  maxEventLoopOverloadedRatio: 0.75,
  maxCpuOverloadedRatio: 0.75,
  maxClientOverloadedRatio: 0.7,
  desiredConcurrencyRatio: 0.5,
  persistStorage: true,
  persistStateIntervalMillis: 10000,
}

const crawler = new PlaywrightCrawler(
  {
    keepAlive: true,
    minConcurrency: 126,
    maxConcurrency: 252,
    maxRequestRetries: 5,
    requestHandlerTimeoutSecs: 10,
    proxyConfiguration,
    useSessionPool: false, // for performance and less detection must be false
    sessionPoolOptions: { maxPoolSize: 426 },
    persistCookiesPerSession: false,
Using linux is now ok
a
@Michael just advanced to level 1! Thanks for your contributions! 🎉
p
Hi @Michael At default crawlee should use only max 25% of your available memory for autoscaling. To increase it you can use:
CRAWLEE_MEMORY_MBYTES
env variable https://crawlee.dev/js/docs/guides/configuration#crawlee_memory_mbytes
2 Views