We did that because broadcastForever is marked with "don't use in production"

I should have explained the use case. We only use loadForever (all replicas) with tables that are infrequently updated and always used in joins. These tables tend to be things like country code data, location (geo) data, etc... The data all fits in a single segment (under 5MM rows). We considered lookup tables but a datasource is significantly easier to work with. I've not deeply tested this. Using broadcastForever rules seem to fit our need but we opted not to use it due to a note about it being experimental. It makes sense in that you would be saving any download time from deep storage (and subsequent deletion of the data) but I imagine it's a small savings. Nonetheless it makes sense for us as queries join those datasources all the time.

A note about JOIN in Druid. All datasources (besides the first) get loaded into memory for the join. The best way to improve performance joining two large tables is to limit the scope of the 2nd table. The best practices I follow are: 1. Ensure your largest datasource is the first table 2. Ensure you always join to a subquery on the right-hand-side 3. Always use a WHERE __time clause to limit segments needed 4. Repeat your WHERE clause conditions inside of all subqueries (as applicable) Example:

SELECT <columns>
FROM datasource1
INNER JOIN (
  SELECT <columns>
  FROM datasource2
  WHERE __time >= <limit the scope of segments to load>
  AND <other conditions as necessary>
) ON <join parameters>
WHERE __time >= <repeated here to limit the scope of datasource1>
AND <other conditions as necessary>

One technique to reduce the number of rows is to use GROUP BY in the subquery,

Generally we try to denormalize our data structures. We are exploring using something along the lines of Apache Spark to pre-join disparate events for Druid consumption.

I'm hoping that Druid adds support for multiple compaction configurations for this reason. Today we use a cron-like tool to execute such jobs via API.