I’m kicking the tyres (tires) of the Configuration Recommendation Engine but I get back the following error:

{
  "_code": 400,
  "_error": "java.lang.IllegalArgumentException: Time column can be only INT or LONG: TIMESTAMP"
}

My input json is:

{
  "schema": {
    "dimensionFieldSpecs": [
      {
        "averageLength": 36,
        "cardinality": 10000,
        "dataType": "STRING",
        "name": "event_id"
      },
      {
        "averageLength": 36,
        "cardinality": 10000,
        "dataType": "STRING",
        "name": "app_id"
      },
      {
        "averageLength": 36,
        "cardinality": 10000,
        "dataType": "STRING",
        "name": "user_id"
      }
    ],
    "dateTimeFieldSpecs": [
      {
        "cardinality": 10000,
        "dataType": "TIMESTAMP",
        "format": "1:MILLISECONDS:EPOCH",
        "granularity": "1:MILLISECONDS",
        "name": "event_at"
      }
    ],
    "metricFieldSpecs": [],
    "schemaName": "app_downloads"
  },
  "queriesWithWeights": {
    "  select count(event_id) as num_downloads from app_downloads where event_at between '2021-01-01 00:00:00' and '2021-01-31 00:00:00' ": 1
  },
  "tableType": "OFFLINE",
  "numRecordsPerPush": 10000,
  "qps": 5,
  "latencySLA": 5000,
  "rulesToExecute": {
    "recommendRealtimeProvisioning": false
  }
}

What could be the problem?

I’ve always used LONG for the dataType, though that was before TIMESTAMP was added. But based on https://docs.pinot.apache.org/basics/components/schema#date-time-fields, Pinot still needs it to be STRING or INT or LONG. Or it could be a bug in the configuration recommender…

@Ken Krugler, thanks for the response. Yes, you are right, according to the docs, the only supported data types for a column in the DateTimeFieldSpec are STRING, INT and LONG. I’ll like to use TIMESTAMP so that end users can use date literals in their SQL statements. Any idea whether TIMESTAMP is supported ? I’m pretty sure I’ve seen examples in the wild where a TIMESTAMP data type is used.

this seems like a validation issue. let me quickly check and see if TIMESTAMP can be added. (as it should)

@Rong R, thanks for the suggestion, unfortunately, changing the timeFormat from epoch to timestamp gave the same error. Looking through the code, configuration recommender generates a sample segment using a data generator to estimate how much memory would be used. The TimeGenerator class doesn’t support a TIMESTAMP data type and hence throws the validation error. Given that a TIMESTAMP is implemented as a LONG, the fix should be trivial. I’ll submit a PR.