This message was deleted.
# generals
Slackbot
10/30/2023, 11:56 PMThis message was deleted.
b
Ben Krug
10/31/2023, 1:38 AMHow big is the dimension table? I'm pretty sure that if it's a broadcast lookup (if it's small enough), this can be done. Not sure about other ways, it might be possible with a combining input source - https://druid.apache.org/docs/latest/ingestion/input-sources#combining-input-source
s
sriramdas sivasai
10/31/2023, 1:40 AMIt’s pretty small of 1000-5000 records
b
Ben Krug
10/31/2023, 2:37 AMIf it's just one key and one value, it might be worth exploring lookup tables. And if you do, I might recommend URI-based over jdbc-based (to avoid jdbc-type issues). I haven't done it, but I have heard a lot about ingestions using lookups to populate values as they ingest.
s
sriramdas sivasai
10/31/2023, 5:32 PMi checked the both the options listed above, but i feel like, both option will not add new columns to streaming table and we can either do joins after the ingestion/lookups or create new datasource using combining datasource option.
b
Ben Krug
10/31/2023, 6:17 PMlookups are a common way to handle dimension tables