This message was deleted.

Hi Inault, IMHO Traditional inner joins is the best and easiest way to do this. If your secondary tables are all relatively small (i.e. < 1m records) then a standard broadcast join may work. But you can also use MSQ sortMerge join to join tables of any size together. Using MSQ SQL I would create a statement something like this:

replace into Foo overwrite all
 select f.__time, f.A, b.C
   from Foo f 
   join Bar b on f.__time = b.__time
partitioned by all

This of course assumes time is the PK for both tables and they are completely matching. If they are not completely matching but still unique, then you can use full outer join and in your select list use

coalesce(f.time, b.time)

instead of just

f.__time

to pull the time field from either table. === An alternate method would be to create a lookup table for Bar and use the lookup() function instead of a regular join ... I don't know how the performance would compare to join though ... and as a lookup you would for sure be subject to the broadcast size limits.

Well, both tables can be quite huge, that's why joins can be a problem. But we can maybe use the MSQ SQL as you mentionned. That seems to be interesting. Thanks !

Fyi: https://druid.apache.org/docs/latest/multi-stage-query/reference#sort-merge

thanks ! Is that quite new in Druid ? I did not seen that on reading druid documentation last year. I may have just missed it.

yes, it's pretty new, released earlier this year ... it might still be in Experimental status ...

it seems to work pretty well on small tables. I'll try on big ones.