hello team, I am trying to use pinot-batch-ingestion to ingest s3 parquet data into pinot. I am getting

java.lang.IllegalArgumentException: INT96 not implemented and is deprecated error

. does someone know, what could be the right way of reading parquet using pinot-parquet record reader? It seems like by default my batch job is using parquetAvroFormatReader and that doesn't implement INT96, what could be other possible way of reading parquet files? stack-trace:

What pinot version are you on? You can try reading the file with

ParquetNativeRecordReader

by setting

useParquetAvroRecordReader

to false in your reader config

I am using

org.apache.pinot:pinot-batch-ingestion-spark:0.10.0
org.apache.pinot:pinot-s3:0.11.0
org.apache.pinot:pinot-parquet:0.11.0

You can try

useParquetAvroRecordReader

= false in your reacordReaderConfigs. Not entirely sure if it'll help, but worth a try. It uses

ParquetNativeRecordReader

with that setting

thanks

Good to know. The next release in default setting will intelligently choose b/w the two readers based on the file metadata so hopefully that should fix this for most users