When a task is submitted in flink on yarn mode, th...
# troubleshooting
p
When a task is submitted in flink on yarn mode, the flink web server is fixed to start. After the above configuration, the port takes effect, but the address is still randomly assigned to the cluster node. Is there any configuration missing, flink1.16.2
d
Consider using YARN Application Router instead for a stable endpoint. Or setting up a reverse proxy server or load balancer that forwards traffic to the dynamically assigned yarn containers ip address & port would be a more stable setup.
p
I want to solve the problem of flink unauthorized access vulnerability, want to fix an ip to start flink web, and then add this machine to the firewall, this method you said is not very understanding
I tried flink1.12.7, according to the above configuration of rest-bind-address can be fixed to an ip to start the web, but version 1.16.2 is not good, I wonder what is the change
@D. Draco O'Brien
d
What is your flink cluster setup. I dont think you said much about that. Is it standalone? Are you on kubernetes etc
I notice that there is both a rest address and a rest.bind-address listed for some 1.16 configs:
Copy code
jobmanager.rpc.address: 0.0.0.0
jobmanager.bind-host: 0.0.0.0
taskmanager.bind-host: 0.0.0.0
taskmanager.host: 0.0.0.0
rest.address: 192.168.100.101
rest.bind-address: 192.168.100.101
Other than that I think the difference is primarily the use of what looks like a dns name as opposed to an actual ip address.
Copy code
rest.bind-address: 0.0.0.0
does this work?
I don’t see that dns is used for the ip bind address in most cases.
Most 1.16 flink installations seem not to be doing this.
My guess is that you previously used a static host file that allowed resolution always to same ip address and you are no longer doing this for the dns name.
p
My running mode is flink on yarn
rest.bind-address: 0.0.0.0 With this setup, the ip will still be random and the port will be the range I configured in the image above
d
alright and when you use another version of Flink? Such as 1.19? what happens then?
Or a version like Flink 1.14 are you seeing a difference?
Well being deployed YARN is a very significant piece of info here.
In a yarn environment the exact ip that flink services like Job Managers rest ip depends on which node manager yarn decides to launch the service on. So you cannot achieve this via flink config alone.
would a list of available ip addresses also be possible or it must be a single static ip address?
The only thing I can see on the Yarn side to control this might be resource manager host names if there is a way to set for the corresponding Flink resource maybe you could control it. But docs only show controlling port ranges not ip address.
p
@D. Draco O'Brien thanks ,I see。I'll try something else