This message was deleted.

I am not aware of any such deployment. What are you trying to achieve by sharing a single topic across clusters?

Think of it like load balancing external traffic to nearest data center. So traffic gets split. Now there is no need to geo replicate this as all pipelines work on local data center clubbing everything downstream

I am assuming this would be for producers only, correct? Consumption wouldn’t make sense in this model.

What do you mean?

A shared topic across clusters would make sense for a producer because it just needs an endpoint to publish to. Consider a scenario where you are trying to collect logs/metrics from multiple servers across the globe. In this case, you want to forward the information to the closest cluster to minimize latency, etc. However, if the closest cluster goes down, you want to fail-over to the next nearest cluster.

Technically, I am not asking for a shared topic. I. E. The partitions of the topic in each local cluster are self contained . Think of it as a replicated global topic, just without the replicators running

But for a consumer, if it fails-over to another cluster, the data that it is looking for won’t be there because it (the data) wasn’t geo-replicated over.

That's applications concern actually. In our case, the produce also moves over and the local backlog of the original cluster is assumed lost until recovery

It sounds like what you need is a solution for topic assignment discovery so all connecting clients can determine which cluster they need to connect to for a given partition.

Maybe we can think about that later.. enhancing the sharding support, but at the moment, requirement is only of a global topic which is present locally in multiple configurable clusters but isn't being replicated among them