This message was deleted.

Hi Bhavok, I haven't done this personally with an OSS Druid cluster ... but if all of the instances old and new are both running in the same cloud with access to the same MetaDB and Deep Storage, then you might be able to just add new servers to your existing cluster, wait for them to come online completely, and then decommission Druid services on the old instances and then shut down those instances and remove them from the cluster. Again I haven't done this with OSS Druid, but in other deployment models this is standard procedure for upgrading instance types. cc @Sergio Ferragut for confirmation or correction on this approach. Thanks. John (edited)

Yes. Decommissioning is available on OSS Druid as well.

@Sergio Ferragut: Thanks for replying. I have a different sort of query. Here what I am trying to 1. Can I point the olf cluster & new cluster to the same S3 deep storage & MySQL without having any issues with data? I want to test new cluster data & see if it is same as old cluster without corrupting data or would I have to completely shut old cluster before that? 2. I saw there are IP addresses in mysql metadata store for tasks table it stores values as host":"172.31.xx.xxx" etc, but new cluster will have different ip address, will this impact the cluster migration?

If there is any data ingestion occurring on both clustered data will get corrupted. If you can keep it without ingestion and you create a second copy of the metadata DB. Then you should be able to read data in the second cluster. The segment_server (I think that’s the name) table does hold cluster specific IPs to keep a map of which segments are replicated on which Historicals, so that’s why you need another MDB. The second cluster will need to load segments into its Historicals before you can query.

@Sergio Ferragut: I am on druid 0.22.1, it doesnot have "segment_server" table in it metadata DB. It has a segments table which holds the s3 uri's & datasource specific information, I can only see IP addresses in "tasks" table in "status_payload" column. It has data as below json blob obj :

{

"id": "single_phase_sub_task_transactions_ekfmjfoj_2023-12-25T09:26:28.220Z",

"status": "SUCCESS",

"duration": 63447,

"errorMsg": null,

"location": {

"host": "172.xx.xx.xxx",

"port": 8102,

"tlsPort": -1

}

}

It has a host which can be seen in above as my private aws ip, so will this have any impact when I point the new cluster table to old copied mysql metadata? I think the deep storage data will be loaded from same old cluster bucket only as s3 uri's are present in "segments" table in "payload" column?