Sushant
02/06/2025, 4:52 PMAll job vertices are currently running at their target parallelism'.
After I do job restart, job scales up to 50 and does scale down to 1.
Message in logs after restart : pendingRecords metric for ***** could not be found. Either a legacy source or an idle source. Assuming no pending records.
job.autoscaler.enabled: "true"
job.autoscaler.scaling.enabled: "true"
job.autoscaler.stabilization.interval: 1m
job.autoscaler.metrics.window: 5m
job.autoscaler.utilization.target: "0.6"
job.autoscaler.target.utilization.boundary: "0.2"
job.autoscaler.restart.time: 2m
job.autoscaler.catch-up.duration: 2m
pipeline.max-parallelism: "720"
#enable in-place scaling
jobmanager.scheduler: adaptive
job.autoscaler.scale-up.grace-period: 2m
job.autoscaler.scale-down.grace-period: 1m
job.autoscaler.scale-down.interval: "2m"Anh Nguyen
04/18/2025, 11:09 PM