Since i never got my answer, asking again once more. The Flink Autoscaler stuff, does anyone has worked with Kinesis as source? I see in docs it works best with Kafka but what about Kinesis? Is it worth spending time to test it? Need suggestions?

I think it would work OK with Kinesis if you use idle shard configuration in the case it scales beyond # of shards. Otherwise you could statically limit the source to # of shards

Are you referring the Kubernetes Operator autoscaling?

Yes i am specifically referring to Kubernetes Operator Scaling.

ok, then no, the Kinesis source does not support it since legacy sources do not emit the required metrics. We are working on a new Source for Kinesis that will support Flink 1.16/1.17. Alternatively @Samrat Deb might be able to help.

This only means it wont scale based on source metrics but still can work with backpressure, busy etc. Or it cannot work overall? Right now we are using CPU/Mem for scaling which TBH is not a very efficient way and we are wasting resources

Yes correct AFAIK

Thankyou for all your reply, i will test this out.