Hi `container` entities allow for documentation tagging owne DataHub #ui

Hi!, `container` entities allow for documentation,...

witty-butcher-82399

07/14/2022, 2:47 PM

Hi!,

container

entities allow for documentation, tagging, ownership, etc. Example: https://demo.datahubproject.io/container/urn:li:container:1f35fc02a28895eabc1b000e3e1ecac8/Entities?is_lineage_mode=false Is that possible for

dataPlatformInstance

entities? (well, is that an entity or still just an aspect 🤔)

echoing-airport-49548

07/14/2022, 3:35 PM

Hi @witty-butcher-82399

dataPlatformInstance

is an aspect. I assume you’re asking about the

dataPlatform

entity! We currently don’t supporting editing these through the UI. Would you mind elaborating what use case you have and why that would be helpful?

witty-butcher-82399

07/14/2022, 3:39 PM

I'm referring to platform instance specifically. Eg: I want to know owner of an instance

echoing-airport-49548

07/14/2022, 4:10 PM

So dataPlatformInstance is just used to say what type of data platform an entity belongs to, say a dataset that comes from Snowflake

echoing-airport-49548

07/14/2022, 4:10 PM

Are you saying that for your use case it would be helpful to have an owner for all Snowflake (just an example) entities?

witty-butcher-82399

07/14/2022, 5:05 PM

Not platform but platform instance. Eg owner of one snowflake instance, or owner of one Kafka cluster,… so users can request access to that particular instance, for example

witty-butcher-82399

07/15/2022, 6:38 AM

I guess my question goes beyond the UI, I will try to provide more context on my request. While many entities have

container

as an aspect, it is also registered as an entity, with many aspects to describe it https://github.com/datahub-project/datahub/blob/baf3f3f33c289c015916c7e460bd22cb4ef74e78/metadata-models/src/main/resources/entity-registry.yml#L109

Copy code

- name: container
    doc: A container of related data assets.
    keyAspect: containerKey
    aspects:
      - containerProperties
      - editableContainerProperties
      - dataPlatformInstance
      - subTypes
      - ownership
      - container
      - globalTags
      - glossaryTerms
      - institutionalMemory
      - browsePaths # unclear if this will be used
      - status
      - domains

The same happens for

dataPlatform

instance: aspect and entity. Here it is the entity definition: https://github.com/datahub-project/datahub/blob/baf3f3f33c289c015916c7e460bd22cb4ef74e78/metadata-models/src/main/resources/entity-registry.yml#L175

Copy code

- name: dataPlatformInstance
    category: internal
    keyAspect: dataPlatformInstanceKey
    aspects:
      - status

However, I miss some aspects describing the instance, such as

ownership

(editableP|p)taPlatformInstanceProperties

, or

institutionalMemory

to name a few. We may also add some aspect or field for the categorization of those instances. In our organization we have about a hundred of different kafka clusters and we want to track the owner for each. Also, those cluster admins may want to add some specific documentation for each cluster. The same for all redshift clusters, hive instances, glue aws accounts, etc we have. So, from my understanding, what’s missing here is: • extending the model for the

dataPlatformInstance

entity • enable the visualization/management of those entities in the UI WDYT?

4 Views

Open in Slack

Previous Next