this feels like such a basic question but what s the right i Apache Flink #troubleshooting

this feels like such a basic question but what’s t...

Trystan

09/25/2023, 10:10 PM

this feels like such a basic question but what’s the right incantation to start multiple TMs locally in docker? adding multiple

localhost

lines to

/opt/flink/conf/workers

doesn’t work (though this used to on some previous versions). the second tm tries to start and gets a port collision.

Copy code

Starting cluster.
Starting standalonesession daemon on host me-5578f564b9-fvzmm.
Starting taskexecutor daemon on host me-5578f564b9-fvzmm.
[INFO] 1 instance(s) of taskexecutor are already running on me-5578f564b9-fvzmm.
Starting taskexecutor daemon on host me-5578f564b9-fvzmm.

cat flink--taskexecutor-1-me-5578f564b9-fvzmm.out

ERROR: transport error 202: bind failed: Address already in use
ERROR: JDWP Transport dt_socket failed to initialize, TRANSPORT_INIT(510)
JDWP exit error AGENT_ERROR_TRANSPORT_INIT(197): No transports initialized [./src/jdk.jdwp.agent/share/native/libjdwp/debugInit.c:735]

but the start up script (invoked through

start-cluster.sh

) clearly supports this. i feel like i must be missing something obvious 🙂

Trystan

10/13/2023, 3:18 PM

in case anyone runs into this in the future the problem was not with the flink setup - it was with the jvm debugger options. an env var was setting

FLINK_ENV_JAVA_OPTS_TM=… , address=*:5005

and it was that debug port binding that prevented the second TM’s startup. not clear what the issue was from the logs but that might be a jvm thing

Open in Slack

Previous Next