Hi Team , What is the timeUnit ( SECONDS, DAYS .....
# troubleshooting
p
Hi Team , What is the timeUnit ( SECONDS, DAYS ..) of
metadataTTL
in upsertConfig ? According to documentation example it is seconds but it is also mentioned
Copy code
Since the metadata TTL is applied on the first comparison column, the time unit of upsert TTL is the same as the first comparison column.
With below conf , ingestion is failing .
Copy code
"segmentsConfig": {
      "schemaName": "agent_task_dimension_v1",
      "retentionTimeUnit": "DAYS",
      "retentionTimeValue": "180",
      "replicasPerPartition": "2",
      "timeType": "DAYS",
      "timeColumnName": "task_created_date" .... }

 "upsertConfig": {
      "mode": "FULL",
      "metadataTTL": 648000
.. }
Copy code
java.lang.ClassCastException: null
pinot-server.log:2025/07/30 23:53:32.481 ERROR [RealtimeSegmentDataManager_agent_task_dimension_v1__16__0__20250730T1823Z] [agent_task_dimension_v1__16__0__20250730T1823Z] Caught exception while indexing the record at offset: 52
I tried bunch of combinations and it is not due to null fields, seems to be due to timeUnit and metadataTTL combination mismatch.
@Pratik Tibrewal, Can you please help when available ?
p
Do you have a comparison column configured for your table?
p
@Pratik Tibrewal , No I haven't added explicit
comparisonColumns
, the primary key columns are
id  and date field
If I add an field having epochTimeStamp as comparisonColumn , then will the
metadataTTL
be in
ms
?
p
Yes metadataTTL is applied based on comparison column. By default, comparison column is the time field. Is the time field currently a date? Like in the format "yyyy-mm-dd" or something?
p
ah ok ..yeah the current time field is date
yyyy-mm-dd
. Its not a epoch timestamp
p
Yeah so metadataTTL expects the value to be of Number type.
p
ok .. I will configure a timestamp field and check . Thanks !
Thanks @Pratik Tibrewal , Ingestion was successful after adding comparisonColumn . If the comparisonColumn is in
seconds
then metadataTTL should be in seconds ..similarly if comparisonColumn captures time in ms then metadataTTL should be in
ms
.. is this correct understanding ?
p
yess
👍 1