I’d like to pull together some stronger recommendations to our team between using pyflink or the Java api. We have many developers and lots of tooling in Python already and none in Java so it would be a fairly substantial point for our organization to take that on. If any folks have experience with both, I’d appreciate input to help with this trade. I’d like to avoid more generic statements like “Java is more performant” since that doesn’t mean python wouldn’t be performant enough. Any anecdotes or advice folks have would be very appreciated!

I would like to know the status of pyflink. In the roadmap it’s marked as beta, whatever that means.

if you use Flink SQL it shouldn't have differences

Related question. Has anyone measured the performance of the DataStream API in PyFlink vs Java/Scala? Even a rough benchmark would be appreciated.

See the benchmarks at the end of this blog post: https://flink.apache.org/2022/05/06/pyflink-1.15-thread-mode.html

@Raf The framework is highly optimized and the cost should be very small. The performance largely depends on what you do(that is, the custom application logic) in the Flink job. For example, you may define a map function in your job and if the map implementation is very efficient, the performance should be very good. However, if the map implementation is very slow, then it will become the bottleneck for the overall performance. That’s, usually the bottleneck doesn’t comes from the framework, but from the application logic.