https://pinot.apache.org/ logo
Join Slack
Powered by
Hi Team, I just installed Pinot locally and I ran ...
# troubleshooting
c
Hi Team, I just installed Pinot locally and I ran the script
Copy code
./bin/quick-start-batch.sh
It is throwing an error:
Copy code
***** Offline quickstart setup complete *****
Total number of documents in the table
Query : select count(*) from baseballStats limit 0
Executing command: PostQuery -brokerHost 127.0.0.1 -brokerPort 8000 -queryType pql -query select count(*) from baseballStats limit 0
Exception in thread "main" java.lang.NullPointerException
	at org.apache.pinot.tools.Quickstart.prettyPrintResponse(Quickstart.java:75)
	at org.apache.pinot.tools.Quickstart.execute(Quickstart.java:174)
	at org.apache.pinot.tools.Quickstart.main(Quickstart.java:207)
This is how the UI looks like
Maybe data is not there..
m
@Neha Pawar any idea ^^
j
There is no live segment on server side
@Cinto Sunny Can you please share the OS and java version you are running with?
c
OS Version: MAC OS 10.15.1 (19B88)
Copy code
java -version
java version "1.8.0_201"
I can try uploading separately…. I assumed the start batch sh will upload some files
x
it will, which binary you are using and download from?
c
apache-pinot-incubating-0.4.0-bin
Pinot server log for reference
pinotServer.log
x
from the log, segment is loaded
m
@Cinto Sunny As @Xiang Fu pointed out, seems like the segment was uploaded. Can you issue the query again and share the broker side log for the query?
c
In the broker logs, I see a new request — RequestId:9, but no data is retrieved in the UI
pinotBroker.log
m
Copy code
2020/07/01 10:48:15.967 INFO [BaseBrokerRequestHandler] [jersey-server-managed-async-executor-11] RequestId:14, table:baseballStats_OFFLINE, timeMs:4, docs:0/0, entries:0/0, segments(queried/processed/matched/consuming/unavailable):0/0/0/0/0, consumingFreshnessTimeMs:0, servers:0/1, groupLimitReached:false, exceptions:0, serverStats:
Seems like 0/1 servers responded
j
Yes, server didn't respond
m
What's the server log snippet around when the query happens?
j
Based on the server log, it didn't even get the query
m
hmm. But is the log from when the query was issued?
j
Might be some security settings on the laptop?
c
The last log from server is 2020/07/01 105623.463
Copy code
2020/07/01 10:56:23.463 INFO [HelixInstanceDataManager] [HelixTaskExecutor-message_handle_thread] Created table data manager for table: baseballStats_OFFLINE
2020/07/01 10:56:23.585 INFO [HelixInstanceDataManager] [HelixTaskExecutor-message_handle_thread] Added segment: baseballStats_OFFLINE_0 to table: baseballStats_OFFLINE
2020/07/01 10:56:23.585 INFO [SegmentFetcherAndLoader] [HelixTaskExecutor-message_handle_thread] Downloaded segment baseballStats_OFFLINE_0 of table baseballStats_OFFLINE crc 3938111273 from controller
j
Copy code
2020/07/01 10:44:52.046 INFO [SegmentFetcherAndLoader] [HelixTaskExecutor-message_handle_thread] Downloaded segment baseballStats_OFFLINE_0 of table baseballStats_OFFLINE crc 3938111273 from controller
2020/07/01 10:56:16.636 INFO [HelixServerStarter] [main] Starting Pinot server
c
The last log from broker is 2020/07/01 130835.302
Copy code
2020/07/01 13:08:35.302 INFO [BaseBrokerRequestHandler] [jersey-server-managed-async-executor-11] RequestId:17, table:baseballStats_OFFLINE, timeMs:4, docs:0/0, entries:0/0, segments(queried/processed/matched/consuming/unavailable):0/0/0/0/0, consumingFreshnessTimeMs:0, servers:0/1, groupLimitReached:false, exceptions:0, serverStats:(Server=SubmitDelayMs,ResponseDelayMs,ResponseSize,DeserializationTimeMs);127.0.0.1_O=2,-1,0,0, query:select AtBatting from baseballStats limit 10
So when the query is fired, the server is not responding
j
Did you re-run the script between 104452 and 105616?
c
Yes, I believe I cancelled and reran the script
not sure of the exact time
I reran because of this issue
Copy code
***** Launch data ingestion job to build index segment for baseballStats and push to controller *****
***** Waiting for 5 seconds for the server to fetch the assigned segment *****
***** Offline quickstart setup complete *****
Total number of documents in the table
Query : select count(*) from baseballStats limit 0
Executing command: PostQuery -brokerHost 127.0.0.1 -brokerPort 8000 -queryType pql -query select count(*) from baseballStats limit 0
Exception in thread "main" java.lang.NullPointerException
	at org.apache.pinot.tools.Quickstart.prettyPrintResponse(Quickstart.java:75)
	at org.apache.pinot.tools.Quickstart.execute(Quickstart.java:174)
	at org.apache.pinot.tools.Quickstart.main(Quickstart.java:207)
j
This could be the same issue where server didn't get the request
c
I am, looking at ZooInspector and I see this
image.png
m
What do you see in external view?
c
What is external view ?
m
See left hand side of zoo inspector image you posted
c
image.png
image.png
m
Hmm, so server is indeed online
c
yep
m
@Jackie What issue are you referring to where server didn't get the request?
Is it something fixed after the release?
fwiw I downloaded the release and ran the same test, and didn't see any issues
@Cinto Sunny can you re-try for good luck 🙂? When you 
ctrl-c
wait for all processess to be cleaned up
I ran on linux though
c
Sure. I can check ps and confirm, then start again
m
Also, are you running from release or from source-code?
c
From release, i downloaded the tar and proceeded with the next steps
m
From source code worked for me on Mac. And release worked for me on linux
let me try downloading the release on Mac
The instruction says
Copy code
# define the pinot version
PINOT_VERSION=0.3.0
that seems wrong, since latest release is 0.4.0. Is that what you set?
c
yes, i set 0.4.0
m
Hmm, I downloaded and tried on Mac, it still works for me
c
hmmm …what options I have to debug..Somebody mentioned security settings on my mac
m
Can you try the source-code path first?
One difference I see between your and my log:
Copy code
,DeserializationTimeMs);127.0.0.1_O=29
yours ^^
mine:
Copy code
DeserializationTimeMs);172.18.168.121_O=2
c
let me try again… as you said earlier …
m
ok
c
must be some minor issue
did you try 0.4.0
m
Yes, I downloaded 0.4.0 and ran on my Mac
c
ok
m
If you are on VPN, try disconnecting (just wild guess)
c
I tried with 0.3.0 and it gives the same error. I also see another error, not sure if they are related
Copy code
Executing command: StartController -clusterName QuickStartCluster -controllerHost 127.0.0.1 -controllerPort 9000 -dataDir /var/folders/v0/1cy26sms2rv0x68whkb0jc3r0000gn/T//PinotController -zkAddress localhost:2123
Invalid instance setup, missing znode path: /QuickStartCluster/CONFIGS/PARTICIPANT/Controller_127.0.0.1_9000
Invalid instance setup, missing znode path: /QuickStartCluster/INSTANCES/Controller_127.0.0.1_9000/MESSAGES
Invalid instance setup, missing znode path: /QuickStartCluster/INSTANCES/Controller_127.0.0.1_9000/CURRENTSTATES
Invalid instance setup, missing znode path: /QuickStartCluster/INSTANCES/Controller_127.0.0.1_9000/STATUSUPDATES
Invalid instance setup, missing znode path: /QuickStartCluster/INSTANCES/Controller_127.0.0.1_9000/ERRORS
This is the entire startup sh logs
pinot_startup_sh.logs
pinot_startup_sh.logs
I guess VPN was the issue.. I disconnected from VPN, killed some running tasks and it seems to be working. Thanks ALL
m
Oh nice. Can you file an issue for the VPN with as much info (logs) as possible? I am on VPN and I didn't see the issue. We had this issue several years ago and thought it was addressed.
@Cinto Sunny ^^
c
Just created a jira. Let me know if anything else is needed. I can update https://issues.apache.org/jira/browse/PINOT-10
m
Oh, we follow github issues. Sorry about that
c
ok..will create one there
Open in Slack
PreviousNext
Powered by