Hello, does lookup loading disabling work for kafk...
# dev
s
Hello, does lookup loading disabling work for kafka task? I see it works for MSQ task.
a
By default for realtime ingestion, all lookups need to be loaded because realtime queries can include any lookup in it.
But I think there's a way to override that if you know that only certain lookups or none are required by setting the context -
lookupLoadingMode
and
lookupsToLoad
(looks like the possible values for mode are
MODE
,
NONE
and
ONLY_REQUIRED
)
s
ya I set it to
NONE
but lookups are loaded still for kafka task
a
I believe in the task logs there should be a log something like
LookupReferencesManager - Loading lookups using spec
that prints the loading mode; btw the docs for this context is here and there's a mention at the top of the table:
The following parameters apply to all task types
Btw this lookup disabling behavior is a bug with streaming ingestion that should be fixed by this PR: https://github.com/apache/druid/pull/18064
s
thats great, thank u!