Alexander Maniates
08/19/2025, 7:09 PMcreateMetadataTarGz for segment creation and preferMetadataTarGz for the metadata push job. In testing this, I ran into errors on the controller side where the DefaultMetadataExtractor expects a segment tar ball to contain a /v3 subdir, so it fails when it receives just the slim tar ball which only contains [metadata.properties, creation.meta] and not the /v3/ subDir
[1] [2] [3] [4]
I have written up a fix that we are testing on our end: https://github.com/apache/pinot/pull/16635
I am curious if we might be doing something wrong on our end, or if folks are using a custom HadoopSegmentCreationMapper that is making a different structured tar ball for the segment metadata? Or are folks implementing a different MetadataExtractor for their own use? We are still on 1.2.0, but it looks like the code hasn't changed much around this.Alexander Maniates
08/19/2025, 7:10 PM