All - We want to load data from a single Kafka top...
# general
a
All - We want to load data from a single Kafka topic to multiple Druid tables with separate filters. Since Druid maintains offsets per supervisor, we believe that there shouldn't be any technical issues with running multiple supervisors consuming from the same topic. Is the understanding correct? We understand that this can introduce undue latency in one or more tables if the data volume on Kafka is skewed - one filter has a disproportionate amount of data, but are there any other issues that we should be concerned about? Thanks, AR.
i
This should be fine with multiple supervisors
b
The primary consideration of that model is that each supervisor will read all of the data and throw out records that do not match the filter. This can be a lot of extra consumption load on kafka if the data volumes are high.