Hi This may be a dumb question.. Why does Flink job manager serialize map functions and send them to task manager? I may have understood the working wrong but basically job manager sends a job to task manager and task manager completes it and reports it back. So, job manager is probably serializing those map and other rich functions and sending them to task manager? If this is right why cant it just say run abc map function with xyz parallelism? Task manager will obviously have the code so it doesnt actually need it to be sent from job manager or maybe I am understanding all this wrong. Can someone clarify the internal working? Thank you

Your understanding is correct. In my understanding Taskmanagers actually don't have the code in Session Mode. Only in Application Mode the Taskmanagers already have the code. cc @Chesnay Schepler

The job manager doesn't send a job to the task manager; but tasks, a unit of work that the jobmanager splits the job into. The problem with your approach is that it would imply running the main method (to actually instantiate the function) on every TM, but this is often not possible because the job may rely on settings/arguments/environment variables that only exist on the client, or it queries external system etc etc.

Ok got it. Thank you so much for the detailed explanation!