Hello, I’m back again.
I’d like to ask about Scali...
# troubleshootingt
Tan Kim
04/28/2023, 2:59 PMHello, I’m back again.
I’d like to ask about Scaling Metric at this time.
When to compute scaling target parallelism, TargetProcessingCapacity need to be calculated, but it doesn’t seem to provide a metric via prometheus reporter from kubernetes operator.
Is there anything I’m missing for this?
Tan Kim
04/28/2023, 3:01 PMI just wanted to show the calculated metric for scaling in dashboard before scaling.
Tan Kim
04/28/2023, 6:02 PMA question unrelated to this, but if the subtasks that exist in the jobgraph all appear to be one, why is that?
In order to do source scaling, the outputRecords value needs to be non-zero, but since the downstream after the kafka source stream is not separated on the jobgraph, the outputRecords is getting zero, so we explicitly added a keyBy operation to the kafka source stream so that we can intentionally separate them and then calculate the outputRecords value.
(I don’t think this is very good for performance) Is there any other way to ensure that the streams are separated into two at the desired location in the jobgraph?