I'm hitting an issue where the offline data I loaded into does not match the query results from Pinot. I run

select * from metrics limit 20000

and I see

numDocsScanned = 15400
totalDocs = 17118

totalDocs matches my raw data size (what I'd expect). The query results match numDocsScanned (which is wrong). I'm happy to share data and schema in a private message.

Are you running the query above as-is, or do you have a filter? If former, then: a) does schema of all segments match? b) are all segments ONLINE in the external view?

I'm running that as-is.

in idealstate or in external view?

It's weird because

numSegmentsQueried=2
numServersResponded=2
numSegmentsProcessed=1
numSegmentsMatched=1
numConsumingSegmentsQueried=1

is this a hybrid table?

It's a hybrid table

if so, perhaps the time column (and unit) is messed up

Interesting

because of that, the offline segment is getting pruned

I'll look into that. Thanks!

👍

Yea, that seems related. If I query just offline, the counts are correct.

select * from metrics_OFFLINE limit 20000

yep

This is on my testing instance. This data is older. There isn't recent offline or realtime data.

i think there's a doc explaining how time boundary is computed. But in essence, for days as time unit, time boundary is max(offlineDay) -1