Hey team, I’ve a partial upsert table and got the following error when building the first segment. Any idea how to solve it?

2022/08/09 03:00:48.603 ERROR [LLRealtimeSegmentDataManager_table_stage__2__0__20220809T0117Z] [table_stage__2__0__20220809T0117Z] Could not build segment
java.lang.IllegalArgumentException: integer overflow detected
	at org.apache.pinot.shaded.com.google.common.base.Preconditions.checkArgument(Preconditions.java:122) ~[pinot-all-0.11.0-SNAPSHOT-jar-with-dependencies.jar:0.11.0-SNAPSHOT-b58810ccf2c7d18693d01688769dcccd3e761d4b]
	at org.apache.pinot.segment.local.segment.creator.impl.fwd.MultiValueVarByteRawIndexCreator.<init>(MultiValueVarByteRawIndexCreator.java:80) ~[pinot-all-0.11.0-SNAPSHOT-jar-with-dependencies.jar:0.11.0-SNAPSHOT-b58810ccf2c7d18693d01688769dcccd3e761d4b]
	at org.apache.pinot.segment.local.segment.creator.impl.DefaultIndexCreatorProvider.getRawIndexCreatorForMVColumn(DefaultIndexCreatorProvider.java:251) ~[pinot-all-0.11.0-SNAPSHOT-jar-with-dependencies.jar:0.11.0-SNAPSHOT-b58810ccf2c7d18693d01688769dcccd3e761d4b]
	at org.apache.pinot.segment.local.segment.creator.impl.DefaultIndexCreatorProvider.newForwardIndexCreator(DefaultIndexCreatorProvider.java:85) ~[pinot-all-0.11.0-SNAPSHOT-jar-with-dependencies.jar:0.11.0-SNAPSHOT-b58810ccf2c7d18693d01688769dcccd3e761d4b]
	at org.apache.pinot.segment.spi.index.IndexingOverrides$Default.newForwardIndexCreator(IndexingOverrides.java:156) ~[pinot-all-0.11.0-SNAPSHOT-jar-with-dependencies.jar:0.11.0-SNAPSHOT-b58810ccf2c7d18693d01688769dcccd3e761d4b]
	at org.apache.pinot.segment.local.segment.creator.impl.SegmentColumnarIndexCreator.init(SegmentColumnarIndexCreator.java:215) ~[pinot-all-0.11.0-SNAPSHOT-jar-with-dependencies.jar:0.11.0-SNAPSHOT-b58810ccf2c7d18693d01688769dcccd3e761d4b]
	at org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl.build(SegmentIndexCreationDriverImpl.java:216) ~[pinot-all-0.11.0-SNAPSHOT-jar-with-dependencies.jar:0.11.0-SNAPSHOT-b58810ccf2c7d18693d01688769dcccd3e761d4b]
	at org.apache.pinot.segment.local.realtime.converter.RealtimeSegmentConverter.build(RealtimeSegmentConverter.java:123) ~[pinot-all-0.11.0-SNAPSHOT-jar-with-dependencies.jar:0.11.0-SNAPSHOT-b58810ccf2c7d18693d01688769dcccd3e761d4b]
	at org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager.buildSegmentInternal(LLRealtimeSegmentDataManager.java:873) [pinot-all-0.11.0-SNAPSHOT-jar-with-dependencies.jar:0.11.0-SNAPSHOT-b58810ccf2c7d18693d01688769dcccd3e761d4b]
	at org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager.buildSegmentForCommit(LLRealtimeSegmentDataManager.java:800) [pinot-all-0.11.0-SNAPSHOT-jar-with-dependencies.jar:0.11.0-SNAPSHOT-b58810ccf2c7d18693d01688769dcccd3e761d4b]
	at org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager$PartitionConsumer.run(LLRealtimeSegmentDataManager.java:699) [pinot-all-0.11.0-SNAPSHOT-jar-with-dependencies.jar:0.11.0-SNAPSHOT-b58810ccf2c7d18693d01688769dcccd3e761d4b]
	at java.lang.Thread.run(Thread.java:829) [?:?]
2022/08/09 03:00:48.607 ERROR [LLRealtimeSegmentDataManager_table_stage__2__0__20220809T0117Z] [table_stage__2__0__20220809T0117Z] Could not build segment for table_stage__2__0__20220809T0117Z

Have you added any multivalue columns to "noDictionaryEnabled" list in your table definition?

Should I or shouldn’t I add?

You shouldn't. Support for non dictionary encoded multi value columns is still a WIP AFAIK.

yes, I’ve added all multivalue columns to “noDictionaryEnabled” list.

That's not supported in any stable versions. https://github.com/apache/pinot/issues/8875 was closed recently but only on master. You can try removing your multi value columns from "noDictionaryEnabled" once

Thanks, @saurabh dubey we’ll try and remove this config.

Good to know @Alice.