A complete solution for open data platforms, enterprise data catalogs, data lakes and data management. Open source, mature, fully-featured and production ready.

DataHub

How to keep pre existed Tags and Owners when re-ingestion a datasource to datahub? Or how to ignored all existed dataset from a datasource when re-ingestion datily datasource to datahub

<@U02JY4YN65S>: for ownership you now have support for PATCH-ing owners so you don’t lose owners in the UI. <https://datahubproject.io/docs/metadata-ingestion/transformers/#advanced-use-case-patching-owners> (available in pypi 0.8.16.4)

For tags, the ingestion tags are kept separate from the tags in the UI already, so you should not be seeing any overwrites today. Please let us know if you are.

+1 Let us know if you are seeing other overwrite cases..

<@U01GCJKA8P9> I saw when new table show up in database, datahub don't auto ingest new table to datahub. So i setup a crontab to ingest daily -&gt; it overwrite my owner + tag

It should definitely not be overwriting tags

From mysql, when ingest a dataset i have a default tag from .yml file, some one change the tags, when crontab re-ingest -&gt; it overwrite tag+owner  to the tag and owner from .yml file

Oh got it - So you're adding a custom transformer to add a tag?

<@U01GCJKA8P9> Yes, i saw the tag + owner overwrite but the *Field Description* doesn't overwrite

My .yml something like this

```source:
  type: mysql
  config:
     host_port: sqldb:3306
     database: database
     username: username
     password: password
     env: PROD
     profiling:
       enabled: true
     # limit: 10000
     table_pattern:
       deny:
         - (.*)bak(.*)
     schema_pattern:
       allow:
         - database
transformers:
  - type: "simple_add_dataset_tags"
    config:
      tag_urns:
        - "urn:li:tag:NeedsDocumentation"
        - "urn:li:tag:my_custom_tag"
  - type: "simple_add_dataset_ownership"
    config:
      owner_urns:
        - "urn:li:corpGroup:PDA"
        - "urn:li:corpuser:datahub"
      ownership_type: "PRODUCER"
sink:
  type: "datahub-rest"
  config:
    server: "<http://datahub:8080>"```

Got it thanks for the details. <@UV0M2EB8Q> does this transformer use the same tags aspect as UI? That’s my guess

<https://github.com/linkedin/datahub/blob/master/metadata-ingestion/src/datahub/ingestion/transformer/add_dataset_tags.py#L49> This transformer does use the Global Tags aspect directly... <@U01JUBAPZPS> Is there another aspect specifically for editable tags?

If not this is expected behavior (though not very useful)

At the entity level, there is only one Global Tags aspect

In order to not overwrite tags and owners written from the UI, you would need to issue a read-modify-write from ingestion

<@U01JUBAPZPS> Can i set read-modify-write mode in .yml file

<@U02JY4YN65S>: we currently have support for patching owners, but don’t yet have support for patching tags. You would change your ownership transformer config like this:
```- type: "simple_add_dataset_ownership"
    config:
      semantics: PATCH
      owner_urns:
        - "urn:li:corpGroup:PDA"
        - "urn:li:corpuser:datahub"
      ownership_type: "PRODUCER"```