Hey team, deployed two different 0.11.0 nightly builds and found that an equality filter predicate on a field with a sorted column index isn’t working as expected if the segment is in CONSUMING state. Ex:

select distinct (company) from role_with_company limit 1000000 -- answer: 51

Queries w/ less than or greater than predicates returns always the correct results:

select count(distinct company) from role_with_company where company < '6269223774083d800011fd95' limit 1000000 -- answer: 36
select count(distinct company) from role_with_company where company > '6269223774083d800011fd95' limit 1000000 -- answer: 14

On the other hand, equality predicates when the segment is in CONSUMING state does not return the correct results:

select count(distinct company) from role_with_company where company = '6269223774083d800011fd95' limit 1000000 -- answer: 0, when segment is in CONSUMING state

When the segment is COMMITTED, the query returns the correct results:

select count(distinct company) from role_with_company where company = '6269223774083d800011fd95' limit 1000000 -- answer: 1, when segment is COMMITTED

Anyone aware of a change in behaviour that was introduced recently? @Richard Startin @Jackie Latest nightly build commit: 0.11.0-SNAPSHOT-438c53b-20220520 Previous nightly build commit: 0.11.0-SNAPSHOT-3403619-20220507

Can you try

select count(distinct company), $segmentName from role_with_company where company = '6269223774083d800011fd95' group by $segmentName limit 1000000

Run the following command to find the segment name for this company:

select distinct ($segmentName), company from role_with_company limit 1000000

Result:

Shall we have a quick zoom?

yes @Jackie