Hello Everyone, We are using Pinot 1.5.1 and are noticing differences with null handling with Boolean columns between single stage and multi stage engines. With single stage engine, with null handling enabled, the query

SELECT activity_name, derived FROM activity WHERE derived = false

does NOT include rows where derived is

null

. When using the multi stage engine, also with null handling enabled, this same query returns rows where

derived

is EITHER

false

OR

null

. Is this expected? Seems like a bug to us.

We’ve also noticed that the same issue can be demonstrated with the single-stage engine. This query, as mentioned above, does NOT include nulls:

SELECT activity_name, derived FROM activity WHERE derived = false

But this query DOES include nulls:

SELECT activity_name, derived FROM activity WHERE NOT derived

I’ve been using Claude to analyze this, and it recommended reporting this as a bug, in case this is helpful: For a Pinot bug report • Version: 1.5.1. • Reproduction: column-based null handling, a nullable

BOOLEAN

column with an inverted index,

SET enableNullHandling=true; SELECT count(*) FROM t WHERE NOT col

. • Symptom: null rows are counted. • Cause:

NotFilterOperator.getNumMatchingDocs()

and

getBitmaps()

invert over all documents without removing nulls, and

FilterOperatorUtils.getNotFilterOperator

returns MatchAll for an empty child without removing nulls. Multistage hits this for ordinary

col = false

filters because Calcite simplifies them to

NOT col

.

Could you please check if this is still in master branch? Please feel free to open a PR :)