It looks like results of the SCAN query type, both...
# general
s
It looks like results of the SCAN query type, both whole query cache and intermediate segment level results, are not cached . Looking at the
getCacheStrategy()
method it doesn't have its own implementation and so is returning null from the default implementation. As a result,
isQueryCacheable()
always returns false for this query type at here. Is there a reason why scan query results are not cached?
Saw one mention here suggesting segment level query results are not cached for scan query type. https://druid.apache.org/docs/latest/querying/query-execution/#datasource-type
I don't see why whole query results cannot be cached
@Gian Merlino - in case you have context on this, is there a reason why result level caching wasn't implemented for scan queries?
On that note, do you see any issues if we were to enable whole result level caching for scan queries but not segment level.
g
sorry, missed this 🧵 at first, but I cannot think of a good reason that caching is not implemented for Scan queries. It's probably just missing an implementation. The main issue to think about would be the potential size of the cached results (Scan query results can be potentially quite large). But, I think that is handled by the caching system generally via the
resultLevelCacheLimit
and
maxEntrySize
parameters in
CacheConfig
s
Thanks for responding, @Gian Merlino. Is missing implementation the reason for not caching intermediate results on data nodes as well?
g
yeah, in general the
CacheStrategy
is responsible for both segment-level and result-level caching (although they use different methods). Currently the Scan query has a null
CacheStrategy
(it does not override
getCacheStrategy
in its
ScanQueryQueryToolChest
) so no caching happens anywhere