Hello Guys it is a basic question "I Know that pinot currently not handling null values curretly " Reading this article what i understand is there is a work around to at least filter by NULL in select statement https://docs.pinot.apache.org/developers/advanced/null-value-support where column IS NOT NULL .. I used to get errors while injesting data from kafka to a table because if this null issue so i modified my table config to have default values for field that might contain null to avoid such an error"not sure if that's an ok practice i did so both in transform injest function and schema default values as follow { "columnName":"special_reference", "transformFunction":"JSONPATHSTRING(json_format(payload),'$.after.special_reference','null')" }, { "columnName":"customer_id", "transformFunction":"JSONPATHLONG(json_format(payload),'$.after.customer_id',-2147483648)" }, ------- schema }, { "name" : "customer_id", "dataType" : "INT", "defaultNullValue": -2147483648 }, { "name" : "user_id", "dataType" : "INT" },{ "name" : "special_reference", "dataType" : "STRING", "defaultNullValue": "null" }, also i added nullhandlingenabled to true "tableIndexConfig": { "loadMode": "MMAP", "nullHandlingEnabled": true what i was expecting that when i filter by is NULL in the query editor it will be able to to map 'null' originated from actual null and return the right query " Am i wrong please help me? I.E select * from next_intentions_trial4 where special_reference IS NOT NULL limit 10 returns record where special_reference null or pinots 'null' i should say I used later version of pinot on kubernetes

do you know an identifier of one of the rows that you expect to be null so that we can see what it's actually returning?

Actually, it is working now. the issue was I had an error in case of null while trying to extract these nullable fields of my json in transform function so I added default which confuse pinot "good thing" between actual null and transformed record ... my question. is there is a good resource for groovy function or way to extract null json ?