https://pinot.apache.org/ logo
#general
Title
# general
c

Carl

07/30/2021, 5:34 PM
Hi, we are ingesting a timestamp data into a offline Pinot offline table as a long data type and 1millisecondsepoch date time specs. But for all null records Pinot ingested as min long value. How should we define the schema so that null timestamp can be also ingested as a null value in Pinot?
m

Mayank

07/30/2021, 5:37 PM
Pinot does not support storing null natively currently. Instead what it provides you is a way to replace nulls with a default value of your choice (can be set in schema). You can then filter those out when querying
However, I am curious what would cause timestamp to be null? This can lead to undesired behavior when it comes to retention (depending on the default value)
c

Carl

07/30/2021, 5:44 PM
This is not the one we use for timecolumnname, it’s one of field happens to be null in certain situation. So could we add e.g. “defaultNullValue” :”null” for this dateTimeFieldSpecs field so it can be filtered out too?
m

Mayank

07/30/2021, 5:53 PM
yes
k

Kamal Chavda

07/30/2021, 6:02 PM
Should the nullHandlingEnabled flag be set to true in the tableIndexConfig for this to work?
m

Mayank

07/30/2021, 7:01 PM
Yeah, you need to provide a value of your choice (must match the data type) and add filter in the query. The
nullHandlingEnabled
flag is not for this.
👍 1