hi, I am trying to source debug the job manager pr...
# troubleshooting
d
hi, I am trying to source debug the job manager process locally without Kubenetes. I have change the start-cluster.sh to have
"$FLINK_BIN_DIR"/jobmanager.sh start-foreground
so I can see what is happening. I have amended flink-console.sh so the last lines are
DEBUG_OPTS=" -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=localhost:9000 "
`exec "$JAVA_RUN" $DEBUG_OPTS $JVM_ARGS ${FLINK_ENV_JAVA_OPTS} "${log_setting[@]}" -classpath "`manglePathList "$FLINK_TM_CLASSPATH:$SQL_GATEWAY_CLASSPATH:$INTERNAL_HADOOP_CLASSPATHS"`" ${CLASS_TO_RUN} "${ARGS[@]}"` . I start the sql-client, create a table and then do a select which hangs, the select normally works without the script alterations. I see in the output of the start-cluster.sh window
2024-06-06 09:58:48,084 INFO org.apache.flink.runtime.resourcemanager.slotmanager.FineGrainedSlotManager [] - Matching resource requirements against available resources.
Missing resources:
Job 4aebd3a4723cb69c643c85ab85688763
ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN}, numberOfRequiredSlots=1}
Current resources:
(none)
2024-06-06 09:58:48,163 INFO org.apache.flink.connector.kafka.source.enumerator.KafkaSourceEnumerator [] - Discovered new partitions: [vivekavro-0]
`2024-06-06 095921,722 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job SELECT
source_1
.
order_id
,
source_1
.
order_time
,
source_1
.`buyer`` `FROM
default_catalog
.
default_database
.
source_1
AS
source_1
(4aebd3a4723cb69c643c85ab85688763) switched from state RUNNING to CANCELLING.` It looks like the addition of the debug options to the job manager java command somehow prevents the a task manager from being initialised! I have tried with in the background and cannot get it to work either. I would be grateful for any pointers.
in job manager foreground mode I can see it would not get to the line in start-cluster.sh that starts the task managers; but in daemon mode this should work. I also tries manually starting a task manager using the script, but this task manager is not found when trying to allocate the slot.
y
Hi David, Seems if you pass
start-foreground
to
jobmanager.sh
, it will be blocked by running
flink-console.sh
which never returns, so the following
TMWorkers start
code never got executed. If you'd like to watch the output of
jobmanager.sh start
and keeps executing the following shell script, just append an
&
to the startup line. Here's my patch:
Copy code
#!/usr/bin/env bash
################################################################################
#  Licensed to the Apache Software Foundation (ASF) under one
#  or more contributor license agreements.  See the NOTICE file
#  distributed with this work for additional information
#  regarding copyright ownership.  The ASF licenses this file
#  to you under the Apache License, Version 2.0 (the
#  "License"); you may not use this file except in compliance
#  with the License.  You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
#
#  Unless required by applicable law or agreed to in writing, software
#  distributed under the License is distributed on an "AS IS" BASIS,
#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#  See the License for the specific language governing permissions and
# limitations under the License.
################################################################################

bin=`dirname "$0"`
bin=`cd "$bin"; pwd`

. "$bin"/config.sh

# Start the JobManager instance(s)
shopt -s nocasematch
if [[ $HIGH_AVAILABILITY == "zookeeper" ]]; then
    # HA Mode
    readMasters

    echo "Starting HA cluster with ${#MASTERS[@]} masters."

    for ((i=0;i<${#MASTERS[@]};++i)); do
        master=${MASTERS[i]}
        webuiport=${WEBUIPORTS[i]}

        if [ ${MASTERS_ALL_LOCALHOST} = true ] ; then
            "${FLINK_BIN_DIR}"/jobmanager.sh start-foreground "${master}" "${webuiport}" & # <= Here!
        else
            ssh -n $FLINK_SSH_OPTS $master -- "nohup /bin/bash -l \"${FLINK_BIN_DIR}/jobmanager.sh\" start ${master} ${webuiport} &"
        fi
    done

else
    echo "Starting cluster."

    # Start single JobManager on this machine
    "$FLINK_BIN_DIR"/jobmanager.sh start-foreground & # <= Here!
fi
shopt -u nocasematch

# Start TaskManager instance(s)
TMWorkers start