Untitled
# troubleshootinge
Elon
07/27/2020, 8:35 PMUntitled
n
Neha Pawar
07/27/2020, 8:40 PM
looks alright to me.
full stack trace?
Neha Pawar
07/27/2020, 8:40 PM
and whats the state of segments in EV?
e
Elon
07/27/2020, 8:41 PMAll the segments are in "CONSUMING" state, there's 1.2m rows that haven't changed since I created the new realtime table.
Elon
07/27/2020, 8:42 PMLooking for the stack trace (stackdriver on google cloud is not straightforward 🙂 seems to filter messages)
Elon
07/27/2020, 8:47 PMAh, got it, unsupported aggregation type "DistinctCount", recreating star tree index without that one
Elon
07/27/2020, 8:47 PM Copy code
"java.lang.IllegalStateException: Unsupported aggregation type: DISTINCTCOUNT
at org.apache.pinot.core.data.aggregator.ValueAggregatorFactory.getValueAggregator(ValueAggregatorFactory.java:60) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-9e7da0349baa23dd02987a3142818dbc6a144fbe]
at org.apache.pinot.core.startree.v2.builder.BaseSingleTreeBuilder.<init>(BaseSingleTreeBuilder.java:148) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-9e7da0349baa23dd02987a3142818dbc6a144fbe]
at org.apache.pinot.core.startree.v2.builder.OffHeapSingleTreeBuilder.<init>(OffHeapSingleTreeBuilder.java:69) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-9e7da0349baa23dd02987a3142818dbc6a144fbe]
at org.apache.pinot.core.startree.v2.builder.MultipleTreesBuilder.getSingleTreeBuilder(MultipleTreesBuilder.java:140) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-9e7da0349baa23dd02987a3142818dbc6a144fbe]
at org.apache.pinot.core.startree.v2.builder.MultipleTreesBuilder.build(MultipleTreesBuilder.java:118) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-9e7da0349baa23dd02987a3142818dbc6a144fbe]
at org.apache.pinot.core.segment.creator.impl.SegmentIndexCreationDriverImpl.buildStarTreeV2IfNecessary(SegmentIndexCreationDriverImpl.java:299) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-9e7da0349baa23dd02987a3142818dbc6a144fbe]
at org.apache.pinot.core.segment.creator.impl.SegmentIndexCreationDriverImpl.handlePostCreation(SegmentIndexCreationDriverImpl.java:261) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-9e7da0349baa23dd02987a3142818dbc6a144fbe]
at org.apache.pinot.core.segment.creator.impl.SegmentIndexCreationDriverImpl.build(SegmentIndexCreationDriverImpl.java:221) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-9e7da0349baa23dd02987a3142818dbc6a144fbe]
at org.apache.pinot.core.realtime.converter.RealtimeSegmentConverter.build(RealtimeSegmentConverter.java:141) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-9e7da0349baa23dd02987a3142818dbc6a144fbe]
at org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager.buildSegmentInternal(LLRealtimeSegmentDataManager.java:746) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-9e7da0349baa23dd02987a3142818dbc6a144fbe]
at org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager.buildSegmentForCommit(LLRealtimeSegmentDataManager.java:697) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-9e7da0349baa23dd02987a3142818dbc6a144fbe]
at org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager$PartitionConsumer.run(LLRealtimeSegmentDataManager.java:604) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-9e7da0349baa23dd02987a3142818dbc6a144fbe]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_262]n
Neha Pawar
07/27/2020, 8:49 PM
did that work in 0.3.0 ? 🤔
😁 1
e
Elon
07/27/2020, 8:50 PMYes
n
Neha Pawar
07/27/2020, 8:50 PM
@Jackie could you take a look at this? ^^
e
Elon
07/27/2020, 8:57 PMSo it looks like in ValueAggregatorFactory the only aggs supported are count, min, max, sum, avg, minmaxrange, distinctcounthll, distinctcountrawhll, precentileest, percentiletdigest
Elon
07/27/2020, 8:57 PMAtleast in the 0.4.0 release
Elon
07/27/2020, 8:58 PMMaybe now it's supported?
Elon
07/27/2020, 8:59 PMIt was working probably because star tree indexes were not created in realtime tables in 0.3.0?
n
Neha Pawar
07/27/2020, 9:00 PM
yeah i was just thinking the same. because in the code it doesnt look like it was ever supported
👍 1
j
Jackie
07/27/2020, 9:21 PM@Elon Here are all the supported aggregations for star-tree: https://docs.pinot.apache.org/basics/features/indexing#index-generation
❤️ 1
e
Elon
07/27/2020, 9:21 PMThanks!
Elon
07/27/2020, 9:23 PMLooks like it never threw an error in 0.3.0 bcz we never actually exercised that part of the code, we only created the specs