hey friends, we are trying to setup pre aggregation for one of realtime tables in our cluster for this schema

{
  "schemaName": "etsyads_metrics",
  "dimensionFieldSpecs": [
    {
      "name": "shop_id",
      "dataType": "LONG"
    },
    {
      "name": "listing_id",
      "dataType": "LONG"
    }
  ],
  "metricFieldSpecs": [
    {
      "name": "cost",
      "dataType": "INT",
      "defaultNullValue": 0
    },
    {
      "name": "impression_count",
      "dataType": "INT",
      "defaultNullValue": 0
    },
    {
      "name": "click_count",
      "dataType": "INT",
      "defaultNullValue": 0
    },
    {
      "name": "order_count",
      "dataType": "INT",
      "defaultNullValue": 0
    },
    {
      "name": "revenue",
      "dataType": "INT",
      "defaultNullValue": 0
    }
  ],
  "dateTimeFieldSpecs": [
    {
      "name": "serve_time",
      "dataType": "LONG",
      "format": "1:SECONDS:EPOCH",
      "granularity": "1:HOURS"
    }
  ]
}

we were reading this https://docs.pinot.apache.org/basics/components/table#pre-aggregation and we set this as our noDictionaryMetrics

"noDictionaryColumns": [
      "click_count",
      "order_count",
      "impression_count",
      "cost",
      "revenue"
    ],
    "aggregateMetrics" : true

but we still see multiple rows for the same dimension values in the same hour, does anyone have some guidance as to why this may be happening?

Is this a realtime table?

that is right

Within each consuming segment there will be unique dimension values

After we truncated our time values to the hour (before sending them to pinot), we did see aggregated values as expected. It almost seemed like pinot was including our time value as a dimension for the purposes of aggregation.

The

granularity

field within the

dateTimeFieldSpecs

is just for description purpose as of now, and pinot won't automatically truncate the time value