Hi here, I have a question about Platform Instance...
# ingestion
b
Hi here, I have a question about Platform InstancesĀ šŸ™‚ A bit of context: in my company we run multiple Kafka Clusters, and in our plans each Cluster would be modelled as a Platform Instance in Datahub, to replace the custom Catalogue api we currently offer to our internal users to list them. However, differently from Platform Instances our Cluster entities have the possibility to add custom properties, to represent cluster configurations, and ownership, a little bit like Containers in Datahub. The issue here is that we fall a bit in a middle ground: on one side Clusters are our own internal abstraction and would be hard to abstract them as a generic concept for the Kafka connector, on the other side the available Platform Instances do not offer enough customisation to fulfil all our needs. In an ideal world (for us), Platform Instances would just allow as much customisation as Containers currently do (and in a sense Clusters represent also physical data containers). Does this make any senseĀ šŸ™‚Ā ? Is there a suggested/possible way we can fully model our Kafka Cluster concept using the current Datahub data model? Thanks!
plus1 2
h
I dont have an answer for you, but I share your problem. šŸ˜… We really ran into this problem during a migration of some topics from one cluster to another. This of course lead to overlapping URNs and chaos. šŸ”„
b
This makes a ton of sense
I think the vision for Platform Instances is exactly this -- a specific, top-level container