https://pinot.apache.org/ logo
Join Slack
Powered by
Hi pinot experts, I tried to upload a segment with...
# troubleshooting
q
Hi pinot experts, I tried to upload a segment with the following command:
Copy code
pinot-admin.sh LaunchDataIngestionJob -jobSpecFile batch_ingestion.yaml
and I can see the output shows it returned 200 and thought it was successful:
Copy code
Start pushing segments: 
...
Response for pushing table enriched_invoices_experiment segment enriched_invoices_experiment_OFFLINE_1595614484_1631217493_0 to location <http://10.124.9.8:9000> - 200: {"status":"Successfully uploaded segment: enriched_invoices_experiment_OFFLINE_1595614484_1631217493_0 of table: enriched_invoices_experiment"}
However, I don’t see it showing up by querying the table. Could you please give some pointers where I could look up the issue? I checked the logs and didn’t find errors.
m
can you check the external view of the table via swagger?
If the segment shows there, then can you make the query to the offline table only (use <tableName>_OFFLINE in your query)?
q
Do you mean checking the segments using this query?
Copy code
curl -X GET "http://<pinot-endpoint>.internal/segments/enriched_invoices_experiment?type=OFFLINE" -H "accept: application/json"
hmm, it doesn’t return anything
oh, you probably meant this one:
Copy code
curl -X GET "<http://pinot-us-central1.cloudkitchens-staging.internal/tables/><tablename>/externalview" -H "accept: application/json"
m
yes
q
it returns no segment from this call:
Copy code
"OFFLINE":{},"REALTIME":null}
m
How about idealstate?
q
the same:
Copy code
✗ curl -X GET "<http://10.124.13.8:9000/tables/enriched_invoices_experiment/idealstate>" -H "accept: application/json"

{"OFFLINE":{},"REALTIME":null}%
I tried uploaded again and I see several logging lines from controller like this :
Copy code
│ istio-proxy [2021-09-19T21:59:31.725Z] "POST /v2/segments?tableName=enriched_invoices_experiment HTTP/1.1" 200 - via_upstream - "-" 7541946 143 14815 105 "-" "Apache-HttpClient/4.5.9 (Java/1.8.0_222)" "934c7966-4e04-48a1-9219-7de02cab41df"  │
│ "10.124.9.8:9000" "10.124.9.8:9000" inbound|9000|| 127.0.0.6:52717 10.124.9.8:9000 10.44.3.130:55565 - default
m
Hmm, your ideal state is empty
May be you don’t have any servers tagged with the tag in table?
q
Copy code
✗ curl -X GET "<http://10.124.13.8:9000/debug/tables/enriched_invoices_experiment/idealstate?verbosity=1>" -H "accept: application/json"
{"code":404,"error":"HTTP 404 Not Found"}%
does it mean the table not exist?
May be you don’t have any servers tagged with the tag in table?
could you please explain more about this?
I can see the segments by this request:
Copy code
✗ curl -X GET "<http://10.124.13.8:9000/segments/enriched_invoices_experiment>" -H "accept: application/json"

[{"OFFLINE":["enriched_invoices_experiment_OFFLINE_1595614484_1631217493_0","enriched_invoices_experiment_OFFLINE_1614584553_1629482988_0"]}]%
feel free to get back tomorrow whenever you are available. it is not urgent..
Hey, it turns out to be that there were some old segments left my temp folder which were uploaded together for the table. I guess that caused some problems internal, though I don’t see any errors.. Thanks for checking
m
So it is working now?
If so can you describe what happened?
q
Yeah, it is working now. I used the same local folder to store segments for different tables. Every time I run the batch ingestion job, it uploaded all the segments to one table which are supposed to be for different tables. I think that caused some issues of showing up the segments for the table. However, I don’t see any error to confirm my guess.
Open in Slack
PreviousNext
Powered by