Apache Druid

yeah, this is currently a deficiency with the nested type (and all complex types)

im working on fixing this in the future, but don’t have an exact timeline, likely druid 27 release

the sad part is it actually does have an index internally on which values are null, just the higher level parts of the query engine are not currently able to take advantage of it yet

on the upside, that means once we do add support, null filtering should be pretty fast even for segments that have already been created

Thanks for explaining.
&gt; null filtering should be pretty fast even for segments that have already been created
That is good to know :+1:

In the meantime, is there a better workaround filter to use? Or what I mentioned seem ok? I’m hoping druid internally stores the json string and `TO_JSON_STRING` is just looking that up instead if serializing it during execution

unfortunately it is serializing it, we store the raw json as smile encoded and then compressed

both of those solutions should work though neither are probably particularly fast unfortunately

If there is a feature request / issue for this please do share. Will be easier for us to follow and know when this gets released.