probably what makes the most sense is to have multi-zone kafka (if you are using kafka for ingest) and single zone druid. typically that's what people do when collecting data from lots of different places into a single location for analytics
Gian Merlino
03/01/2023, 2:09 AM
if you need multi-zone druid for DR, you can have the two druid zones read the same kafka data independently (i.e. no shared druid infra)
Are you referring to Availability Zones? Or Regions? If the latter, then will something like Decodables work here?
b
Bharat Thakur
03/01/2023, 7:21 AM
@can I have same s3 bucket for multiple independent Druid
j
John Kowtko
03/02/2023, 3:19 AM
According to the documentation, if you specify a different druid.storage.baseKey value for each cluster, then the files in the same S3 bucket will be kept separate from each other.
ref: https://druid.apache.org/docs/latest/development/extensions-core/s3.html#deep-storage
I do not know if there are any other common files that might conflict if multiple clusters tried to use the same S3 bucket ... but the docs do not mention any restrictions here.