This message was deleted.

It’s not possible to run ingestion from segments (yet). Easiest and recommended path is to replicate the source data and ingest into both. Replication of the metadata and segment files can be done but I would recommend a log based cdc process to avoid load on the metadata db.

We have done something similar in the past, but like you ran into issues with the metadata db. If you’re going to do this on a recurring basis, you could set up a job to connect to your production database and create a batch ingest job from the database rather than trying to change or copy the underlying data. Additionally, if your data providers support it, you could do parallel ingests (e.g. 1 kafka stream to 2 separate druid clusters)

Thank you guys!