Hi, we are ingesting a timestamp data into a offline Pinot offline table as a long data type and 1millisecondsepoch date time specs. But for all null records Pinot ingested as min long value. How should we define the schema so that null timestamp can be also ingested as a null value in Pinot?
07/30/2021, 5:37 PM
Pinot does not support storing null natively currently. Instead what it provides you is a way to replace nulls with a default value of your choice (can be set in schema). You can then filter those out when querying
However, I am curious what would cause timestamp to be null? This can lead to undesired behavior when it comes to retention (depending on the default value)
07/30/2021, 5:44 PM
This is not the one we use for timecolumnname, it’s one of field happens to be null in certain situation. So could we add e.g. “defaultNullValue” :”null” for this dateTimeFieldSpecs field so it can be filtered out too?
07/30/2021, 5:53 PM
07/30/2021, 6:02 PM
Should the nullHandlingEnabled flag be set to true in the tableIndexConfig for this to work?
07/30/2021, 7:01 PM
Yeah, you need to provide a value of your choice (must match the data type) and add filter in the query. The