Pinot is creating all these directories for the segments of about 450 MB each. Do we need to keep it while spark submit load job is still in progress or can i delete them before job completion.. wondering if its still needed in further steps.. its consuming too much space on disk especially for local mode run

Hmm, seems like an issue cc: @Kartik Khare

On it. For some reason we are trying to copy plugins for each input file. Will change that in code.

Thanks @Kartik Khare