Weixiang Sun
12/14/2021, 5:29 PMKen Krugler
12/14/2021, 6:35 PMWeixiang Sun
12/14/2021, 9:39 PMWeixiang Sun
12/15/2021, 1:17 AMElon
12/15/2021, 1:23 AMElon
12/15/2021, 1:23 AM/**
* This method takes a map of segment downloadURI to corresponding tar file path, and push those segments in metadata mode.
* The steps are:
* 1. Download segment from tar file path;
* 2. Untar segment metadata and creation meta files from the tar file to a segment metadata directory;
* 3. Tar this segment metadata directory into a tar file
* 4. Generate a POST request with segmentDownloadURI in header to push tar file to Pinot controller.
*
* @param spec is the segment generation job spec
* @param fileSystem is the PinotFs used to copy segment tar file
* @param segmentUriToTarPathMap contains the map of segment DownloadURI to segment tar file path
* @throws Exception
*/
Elon
12/15/2021, 1:24 AMWeixiang Sun
12/15/2021, 1:25 AMElon
12/15/2021, 1:27 AMWeixiang Sun
12/15/2021, 1:28 AMElon
12/15/2021, 1:32 AMElon
12/15/2021, 1:33 AMKen Krugler
12/15/2021, 10:01 PMElon
12/15/2021, 10:02 PM