Hi ! what could be the possible reason for the bel...
# troubleshootingp
Prakhar Pande
10/18/2022, 11:18 AMHi ! what could be the possible reason for the below error in controller logs.
Copy code
Caught 'java.net.ConnectException: Connection timed out (Connection timed out)' while executing: GET on URL: <http://100.64.49.183:8097/table/catalog_views_test_REALTIME/size>
Caught 'java.net.ConnectException: Connection timed out (Connection timed out)' while executing: GET on URL: <http://100.64.4.104:8097/table/catalog_views_test_REALTIME/size>
Connection errorx
Xiang Fu
10/18/2022, 6:02 PM
can you ping that ip?
p
Prakhar Pande
10/19/2022, 5:18 AM% ping 100.64.4.104
PING 100.64.4.104 (100.64.4.104): 56 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1
Request timeout for icmp_seq 2
Request timeout for icmp_seq 3
Request timeout for icmp_seq 4
Request timeout for icmp_seq 5
.
.
.
Prakhar Pande
10/19/2022, 6:09 AMgot it, it is because the particular server is dead.
but this is happening when I am redeploying pinot while keeping the same zookeeper which is in different namespace. What I am expecting is that the old data will be available again as metadata is there in zookeeper, but the table is going in bad state and download is not happening. Moreover the below is also there in the controller logs
Copy code
Finished reading information for table: catalog_views_test_REALTIME with 2/6 server responses
Missing size report for 12 out of 16 segments for table catalog_views_test_REALTIMEPrakhar Pande
10/19/2022, 6:12 AMMoreover when I am clicking on any server, broker or controller in the ui whether dead or alive , the page is taking a lot of time to load.
x
Xiang Fu
10/20/2022, 1:12 AM
How did you specify the zookeeper in controller config?
Xiang Fu
10/20/2022, 1:12 AM
Did you give the DNS resolvable name?
Xiang Fu
10/20/2022, 1:13 AM
Can you check what’s the download url for those segments?
Xiang Fu
10/20/2022, 1:13 AM
Are they on deep store like s3?
p
Prakhar Pande
10/20/2022, 9:58 AMI have specified zookeeper in the format <service-name>.<namespace>.svc.cluster.local:2181
yes the download url is there and it is of s3 and it is what I specified.
x
Xiang Fu
10/20/2022, 10:04 AM
so those missing segments are on s3? Can you try to tail the controller and server log and meanwhile reload a missing segment? Wanna see which part throws the error