We can upload a flamegraph. <@U019A4U757C> is the ...
# troubleshootinge
Elon
08/21/2020, 7:34 PMWe can upload a flamegraph. @Onha Choe is the brave person that is working with the pinot logging controller. Anyone familiar with this that we can work with?
Elon
08/21/2020, 7:34 PMHere is the flamegraph
Elon
08/21/2020, 7:35 PMflamegraph.svg
Elon
08/21/2020, 7:35 PM@Onha Choe has some jstacks and other context as well.
Elon
08/21/2020, 7:36 PMflamegraph2.svg
o
Onha Choe
08/21/2020, 7:36 PM^ Seems like controller is responsible for grabbing segments and compress/upload to object store(GCS) via pinotFS. Not sure if it is necessary for this to be going through the controller, or can this be uploaded directly from servers
e
Elon
08/21/2020, 10:22 PMLooks like there is a call to do this on the server
Elon
08/21/2020, 10:23 PMAlso, it seems like LLCSegmentCompletionHandler.segmentCommit() should respond asynchronously so that the threads don't wait on io
Elon
08/21/2020, 10:36 PM@Onha Choe - want to try this?
Copy code
listener.getTransport().setIOStrategy(WorkerThreadIOStrategy.getInstance());n
Neha Pawar
08/23/2020, 12:51 AM
@Ting Chen ^^
t
Ting Chen
08/25/2020, 6:36 PMIn the deep store by pass (see the cwiki for design), servers can directly upload segments to deepstore.
o
Onha Choe
08/28/2020, 12:54 AM@Ting Chen I’ve looked at the cwiki, though non of the PR were POCs and never got merged? What is the status on it?
t
Ting Chen
08/28/2020, 1:01 AMhttps://github.com/apache/incubator-pinot/pulls?q=is%3Apr+author%3Achenboat+is%3Aclosed those PR marked as deepstore by-pass are releated.
👍 1
Ting Chen
08/28/2020, 1:02 AMThere is a last PR for integration test and bug fix waiting for approval now : https://github.com/apache/incubator-pinot/pull/5857
n
Neha Pawar
08/30/2020, 8:02 PM
FYI @Kishore G
Neha Pawar
08/30/2020, 8:02 PM
@Ting Chen is this part already implemented:
In the deep store by pass (see the cwiki for design), servers can directly upload segments to deepstore.Neha Pawar
08/30/2020, 8:02 PM
whats the configuration to achieve this (servers uploading to deep store) ?
t
Ting Chen
08/31/2020, 5:21 PM#5314 implements server direct upload to the deep store.
Ting Chen
08/31/2020, 5:25 PMthere are two new config to enable that:
Ting Chen
08/31/2020, 5:25 PM1. pinot.server.instance.segment.store.uri for deepstore uri
Ting Chen
08/31/2020, 5:26 PM2. a table config segmentsValidationAndRetentionConfig.setPeerSegmentDownloadScheme()
Ting Chen
08/31/2020, 5:26 PMI will add them to the doc.