Aaron Wishnick
05/07/2021, 8:29 PMjava.io.IOException: Could not read footer: java.io.IOException: Could not read footer for file DeprecatedRawLocalFileStatus{path=file:/tmp/pinot-7dd1e9e9-b1bd-416c-ab
4b-1e66a887d7ca/input/20210507_201917_26196_rv8nu_14b41b59-66e3-4f97-9df0-56c76d859102; isDirectory=false; length=3804; replication=1; blocksize=33554432; modification_time=1620419283015; access_time=0; owner=; group=; permission=rw-rw-rw-; isSymlink=false}
at org.apache.parquet.hadoop.ParquetFileReader.readAllFootersInParallel(ParquetFileReader.java:248) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-7ac8650777d6b25c8cae4ca1bd5460f25488a694]
Caused by: java.io.IOException: can not read class org.apache.parquet.format.FileMetaData: Required field 'codec' was not present! Struct: ColumnMetaData(type:INT32, e
ncodings:[BIT_PACKED, PLAIN_DICTIONARY, RLE], path_in_schema:[date], codec:null, num_values:2, total_uncompressed_size:54, total_compressed_size:72, data_page_offset:4
, statistics:Statistics(max:7F 62 34 01, min:7F 62 34 01, null_count:0), encoding_stats:[PageEncodingStats(page_type:DICTIONARY_PAGE, encoding:PLAIN_DICTIONARY, count:
1), PageEncodingStats(page_type:DATA_PAGE, encoding:PLAIN_DICTIONARY, count:1)])
Mayank
Aaron Wishnick
05/07/2021, 8:45 PMMayank
Aaron Wishnick
05/07/2021, 8:53 PMMayank
Aaron Wishnick
05/07/2021, 9:01 PMMayank
Aaron Wishnick
05/07/2021, 9:22 PMMayank
Aaron Wishnick
05/07/2021, 9:29 PMMayank
Aaron Wishnick
05/07/2021, 9:31 PMMayank
Xiang Fu
org.apache.pinot.plugin.inputformat.parquet.ParquetAvroRecordReader
?org.apache.pinot.plugin.inputformat.parquet.ParquetNativeRecordReader
Aaron Wishnick
05/07/2021, 10:45 PMorg.apache.pinot.plugin.inputformat.parquet.ParquetRecordReader
(I think that's what the docs said to do)Xiang Fu
Aaron Wishnick
05/10/2021, 2:07 PMXiang Fu
Mayank