This message was deleted.

what version are you on? I know there were vectorization improvements in the 24.0 release https://github.com/apache/druid/releases#24.0.0-query-engine-updates-query-performance-improvements

the only time i’m aware of off the top of my head that non-vectorized can be sometimes a decent amount faster than vectorized is with certain expression virtual columns involving a “single input” column expression, such as some transform on __time, and where the value cardinality of that column within the segment is very low. this is because the non-vectorized expression processing stuff has a cache that it uses for single input long and string expressions that has not been added to the vectorized processing yet

if you have an example of a query that is slower with vectorization enabled, i'd love to see it

Hello, here is a trimmed down example where vectorization is much slower. In this example, vectorization turned on hits a timeout at 300s. w/ it turned off, it finishes < 8s or so. Druid version: 0.23 datasource stats: • total segments: 39,593 • avg segment size: ~350mb and ~5.9million rows • segment granularity: week • total segments queried: 282 Some observations: • querying 1 dimensions works regardless of whether vectorization is on or off. • w/ vectorization on, it fails as soon as a second dimension is added. • if i’m looking at things correctly, the ttfb and query cpu usage is much larger when vectorization is on.

col1

has a cardinality of 6 and

col2

has a cardinality of 352 w/ the query returning 1176 total rows.

{
  "queryType": "groupBy",
  "dataSource": {
    "type": "union",
    "dataSources": [
      "my-data-source"
    ]
  },
  "granularity": "all",
  "intervals": [
    "2022-11-27T00:00:00+00:00/2023-01-01T00:00:00+00:00"
  ],
  "dimensions": [
    "col1",
    "col2"
  ],
  "limitSpec": {
    "type": "default",
    "limit": 30000,
    "columns": [
      "col1",
      "col2"
    ]
  },
  "aggregations": [
    {
      "type": "longSum",
      "name": "longSum(count)_0x0b",
      "fieldName": "count"
    }
  ],
  "filter": null,
  "having": null,
  "context": {
    "queryId": "my-test-query",
    "useCache": true,
    "populateCache": true,
    "useResultLevelCache": true,
    "populateResultLevelCache": true,
    "maxOnDiskStorage": 0,
    "priority": 20,
    "vectorize": true,
    "debug": true
  }
}