Hi team,
We have a realtime table in Pinot cluster.
We want to migrate the realtime data to another Pinot cluster. is it possible to load the COMPLETED segments into new cluster somehow?
Pinot version: 1.0
m
Mayank
06/11/2025, 2:34 PM
You will need to write a script to call the upload api and push the new segments
a
Anish Nair
06/11/2025, 4:41 PM
are you referring to this?
m
Mayank
06/11/2025, 4:46 PM
Yes
a
Anish Nair
06/12/2025, 10:07 AM
@Mayank@Xiang Fu, one more doubt, do we need to keep the no. of kafka topic partition same in new cluster?
Anish Nair
06/12/2025, 10:28 AM
also how to make sure that the CONSUMING segment and the uploaded segment is part of the same server.
Since real time table requires topic partition to server mapping to be maintained.
m
Mayank
06/12/2025, 1:41 PM
Is this upsert enabled?
Mayank
06/12/2025, 1:42 PM
You can push segments to upsert enabled table, it will maintain the upsert guarantees needed.
a
Anish Nair
06/12/2025, 2:08 PM
yeah, this is upsert enabled table
x
Xiang Fu
06/12/2025, 3:20 PM
Then you should keep all the partitions same for kafka and pinot table