https://pinot.apache.org/ logo
Join Slack
Powered by
```protected IntermediateResultsBlock getNextBlock...
# troubleshooting
d
Copy code
protected IntermediateResultsBlock getNextBlock() <--- here?
s
You said indexedTable.finish() was throwing NPE
So my suggestion is to put breakpoint inside the function and step through it
d
yes that call is inside getNextBlock
s
yes so what happens when you go inside finish?
d
the problem is not the finish() method is _indexedTable that is null
s
Why is the indexedTable not getting initialized in this code?
Copy code
_initLock.lock();
            try {
              if (_dataSchema == null) {
                _dataSchema = intermediateResultsBlock.getDataSchema();
                _indexedTable = new ConcurrentIndexedTable(_dataSchema, _brokerRequest.getAggregationsInfo(),
                    _brokerRequest.getOrderBy(), _indexedTableCapacity);
              }
            } finally {
              _initLock.unlock();
            }
d
yeah i am trying to debug that piece of code
one moment
ok hit that line of inizialization
s
I wonder why the NPE was not caused here
Copy code
_indexedTable.upsert(key, record);
d
but it remain null
then i get an exception
Could not find column drawdown(id,amount) in data schema <--
why?
s
Where are you getting this message from?
Can you try the query without aliasing once?
Try
Copy code
SELECT drawdown(id, amount) FROM table GROUP BY strategy_id ORDER BY drawdown(id, amount)
d
ok i try without aliasing
i get that error in line 187
i get that exception
Copy code
} catch (Exception e) {
  LOGGER.error("Exception processing CombineGroupByOrderBy for index {}, operator {}", index,
      _operators.get(index).getClass().getName(), e);
  mergedProcessingExceptions.add(QueryException.getException(QueryException.QUERY_EXECUTION_ERROR, e));
i try without aliasing
wait how can i order with that?
i mean the problem is because i am using the ORDER BY clause too
s
Copy code
SELECT drawdown(id, amount) FROM table GROUP BY strategy_id ORDER BY drawdown(id, amount)
d
if i do not use ORDER BY i do not get any error
ah ok
repeating the aggregator
one moment
s
Where are you getting this message from?
Copy code
Could not find column drawdown(id,amount) in data schema <--
d
line 187 CombineGroupByOrderByOperator.java
s
that is where it is reported/caught
who throws it
I strongly suggest to step through each and every line of code and then step into
to identify the root cause of the exception
d
yes ok
but now i test it without alias
one moment
then i check again to understand who throws it
grr blackout one moment
i am ingesting the data again
@Sidd same error: Could not find column drawdown(id,amount) in data schema
i see that exception after: _initLock.unlock();
s
This is coming from TableResizer
Put a breakpoint in the constructor of TableResizer
the exception of missing column name in data schema is coming from there
d
ok
Copy code
if (columnIndexMap.containsKey(column)) {
  int index = columnIndexMap.get(column);
  if (index < numKeyColumns) {
    _orderByValueExtractors[orderByIdx] = new KeyColumnExtractor(index);
  } else {
    AggregationFunction aggregationFunction = aggregationColumnToFunction.get(column);
    _orderByValueExtractors[orderByIdx] = new AggregationColumnExtractor(index, aggregationFunction);
  }
} else {
  throw new IllegalStateException("Could not find column " + column + " in data schema");
}
Copy code
if (columnIndexMap.containsKey(column))  <-- from here
that else throws it
s
I know
Why is not being added to the map?
Copy code
for (int i = 0; i < numColumns; i++) {
      String columnName = dataSchema.getColumnName(i);
      columnIndexMap.put(columnName, i);
What are all the column names in data schema as retrieved in the above loop?
d
i try to inspect that map one moment
columns from there
0 = "strategy_id" 1 = "drawdown(amount)"
thats the problem
it miss id,
s
there you go
d
id,amount
eh, why?
DataSchema dataSchema
why does the id is missing?
s
We need to go back to CombineGroupByOrderByOperator
where the data schema is being created
Put breakpoint at lin #89 in AggregationGroupByOrderByOperator
This is where it is being created and then being used in combine
you want to do team viewer?
d
not this CombineGroupByOrderByOperator ?
we were working on that
i will back in 30 minutes... (have dinner)
however we were working on that no?
s
data schema is being created in AggregationGroupByORderByOperator
it returns IntermediateResultsBlock with a data schema
d
Copy code
_dataSchema = intermediateResultsBlock.getDataSchema();
^ here right ?
s
yes that's what I am saying.. this intermediate block is returned by a segment level operator -- AggregationGroupByORderByOperator
put a breakpoint in its constructor
What is the impelmentation of this function for your aggregation function?
Copy code
String getResultColumnName();
d
Copy code
public String getResultColumnName() {
  return getType().getName().toLowerCase() + "(" + _column + ")";
}
s
That's the problem
d
what should it be? something like _firstcol UNDERSCORE _secondcol ?
s
It should be the signature of the function with exact arguments both id and amount
At least for now it is needed/assumed by the order by operator when it creates the schema
d
ok but do i need a separator?
i mean, in that example it is name(name)
in mine? with two columns ?
s
Yes
d
what should i return ? what signature? i mean how can i combine two columnnames?
s
drawdown (id, amount)
d
ah
got
it
i iwll try that way
in few minutes! thank you really much @Sidd pardon for the fast/quick (and with errors messages) but i am having dinner right now 🙂 eheh
thank you! i will try it shortly
@Sidd ok the error is gone, but now i get:
"message": "QueryExecutionError:\njava.lang.ClassCastException: org.apache.pinot.core.query.aggregation.function.customobject.RangeSet cannot be cast to java.lang.Comparable\n\tat org.apache.pinot.core.data.table.TableResizer$AggregationColumnExtractor.lambda$new$0(TableResizer.java:311)\n\tat org.apache.pinot.core.data.table.TableResizer$AggregationColumnExtractor.extract(TableResizer.java:320)\n\tat org.apache.pinot.core.data.table.TableResizer.getIntermediateRecord(TableResizer.java:138)\n\tat org.apache.pinot.core.data.table.TableResizer.convertToIntermediateRecordsPQ(TableResizer.java:183)\n\tat org.apache.pinot.core.data.table.TableResizer.resizeRecordsMap(TableResizer.java:159)\n\tat org.apache.pinot.core.data.table.ConcurrentIndexedTable.resize(ConcurrentIndexedTable.java:141)\n\tat org.apache.pinot.core.data.table.ConcurrentIndexedTable.finish(ConcurrentIndexedTable.java:174)\n\tat org.apache.pinot.core.operator.CombineGroupByOrderByOperator.getNextBlock(CombineGroupByOrderByOperator.java:208)\n\tat org.apache.pinot.core.operator.CombineGroupByOrderByOperator.getNextBlock(CombineGroupByOrderByOperator.java:60)\n\tat org.apache.pinot.core.operator.BaseOperator.nextBlock(BaseOperator.java:49)\n\tat org.apache.pinot.core.operator.InstanceResponseOperator.getNextBlock(InstanceResponseOperator.java:37)\n\tat org.apache.pinot.core.operator.InstanceResponseOperator.getNextBlock(InstanceResponseOperator.java:26)\n\tat org.apache.pinot.core.operator.BaseOperator.nextBlock(BaseOperator.java:49)\n\tat org.apache.pinot.core.plan.GlobalPlanImplV0.execute(GlobalPlanImplV0.java:48)"
RangeSet is my object, does it must implement Comparable?
s
Yes
d
ok
Open in Slack
PreviousNext
Powered by