If I have a non keyed data stream piped into a process funct Apache Flink #troubleshooting

If I have a non-keyed data stream piped into a pro...

Nathanael England

01/17/2023, 12:07 AM

If I have a non-keyed data stream piped into a process function with

.process

, that will be run in parallel, right? I don't need to maintain any state in this and the only discussion I'm seeing on parallelism restrictions are when you want to do windowing.

David Anderson

01/17/2023, 1:19 AM

Yes, that will run in parallel.

David Anderson

01/17/2023, 1:20 AM

Same is true for map, flatmap, filter.

David Anderson

01/17/2023, 1:21 AM

A non-keyed window can’t run in parallel because it has to compute a global aggregation, across all of the data.

👍 1

Nathanael England

01/17/2023, 1:23 AM

Great, thanks for confirming!

Open in Slack

Previous Next