Praneeth G
07/31/2025, 6:03 AMmetadataTTL in upsertConfig ? According to documentation example it is seconds but it is also mentioned
Since the metadata TTL is applied on the first comparison column, the time unit of upsert TTL is the same as the first comparison column.
With below conf , ingestion is failing .
"segmentsConfig": {
"schemaName": "agent_task_dimension_v1",
"retentionTimeUnit": "DAYS",
"retentionTimeValue": "180",
"replicasPerPartition": "2",
"timeType": "DAYS",
"timeColumnName": "task_created_date" .... }
"upsertConfig": {
"mode": "FULL",
"metadataTTL": 648000
.. }
java.lang.ClassCastException: null
pinot-server.log:2025/07/30 23:53:32.481 ERROR [RealtimeSegmentDataManager_agent_task_dimension_v1__16__0__20250730T1823Z] [agent_task_dimension_v1__16__0__20250730T1823Z] Caught exception while indexing the record at offset: 52
I tried bunch of combinations and it is not due to null fields, seems to be due to timeUnit and metadataTTL combination mismatch.Praneeth G
07/31/2025, 6:19 AMPratik Tibrewal
07/31/2025, 8:18 AMPraneeth G
07/31/2025, 8:30 AMcomparisonColumns , the primary key columns are id and date field
If I add an field having epochTimeStamp as comparisonColumn , then will the metadataTTL be in ms ?Pratik Tibrewal
07/31/2025, 8:31 AMPraneeth G
07/31/2025, 8:32 AMyyyy-mm-dd . Its not a epoch timestampPratik Tibrewal
07/31/2025, 8:33 AMPraneeth G
07/31/2025, 8:33 AMPraneeth G
07/31/2025, 9:57 AMseconds then metadataTTL should be in seconds ..similarly if comparisonColumn captures time in ms then metadataTTL should be in ms .. is this correct understanding ?Pratik Tibrewal
07/31/2025, 10:26 AM