This message was deleted.

Same issue with strings also

Hi @tilak chowdary, I am trying to reproduce this on my local test install ... can you provide a bit more info: 1. select count(*), count(distinct originEntityIds) from TraceView where traceId = '<your GUID>' a. am expecting 35m for count(*), not sure about the count(distinct) 2. Confirm table schema matches the select list: a. __time b. traceId: string c. originEntityIds: string d. numErrors: number 3. Have you tried running just the SELECT query without the insert to see if it runs to completion? a. I see your first screenshot looks like a sample select but does not match the statement in the SQL buffer shown above it, and is only one record of output. Thanks. John

1,3 attached screenshots, 2, Yes,

"dimensions": [
          {
            "type": "string",
            "name": "traceId",
            "multiValueHandling": "SORTED_ARRAY",
            "createBitmapIndex": true
          }
          {
            "type": "string",
            "name": "originEntityIds",
            "multiValueHandling": "SORTED_SET",
            "createBitmapIndex": true
          },
        ],"metricsSpec": [

        {
          "type": "longMax",
          "name": "numErrors",
          "fieldName": "numErrors"
        }
      ],

Okay, so the row count isn't high and the query only works fine. I'm trying to come up with a test case here... One thing I noticed is that you don't have a corresponding __time filter on the query that is feeding the replacement of the one hour time chunk ... is that intentional?

It wasn't intentional, anyway I have added it and no change in the result

Okay I am able to reproduce what you are seeing, on some data I extracted from the wikipedia demo data set. I created this test table using STRING_TO_MV(), I'm going to check to see if that was enough or if it has to be in array format to begin with ...

I hit the issue with simple strings, I mean not multivalues

Fyi the select version of this query works fine ... I was trying to see if there was an issue related to whether or no the data started out as an array ... looks like that doesn't make a difference.

@John Kowtko please share the bug number, I would like to watch

HI @tilak chowdary, I logged an internal JIRA ticket (I'm an Imply employee new to Druid) which I am assuming you will not be able to see ... I am checking on where to file the bug for community visibility, will let you know when I have a link. Thanks.

druid doesn’t currently have actual array typed columns for segments so it is unable to store the output of this aggregation (and other array typed functions).

cc: @meili @Bhanu Kovuri

@tilak chowdary can you try setting the context

finalizeAggregations:true

in your MSQ insert query and see if it works ?

@Karan Kumar Thanks, It did work, Can you elaborate why do we need that flag? Any side effects

So when you set

finalizeAggregations:true

you are basically telling sql based ingestion I donot want to roll up more. This means that we donot store the agg factory information as part of the segment metadata. The exception you were getting was from the segment generation face where we are trying to store this aggregator factory information while generating the segment. As the calcite layer now handles returns arrays<String> we would need to tell the segmentGeneratorFrameProcesser how to handle such agg factories. Can you please raise a github issue for this. I think this is worthwhile to track