Hi all, is there any way in kafka Ingestion recipe...
# ingestion
a
Hi all, is there any way in kafka Ingestion recipe to add owner and domain. Thank you!
c
Please take a look at transformers
a
@careful-pilot-86309 Can I put it in kafka recipe? just adding transformers section to the recipe?
c
yes
a
Greate, thank you!
@careful-pilot-86309 where exactly i add the transformers in the kafka recipe because its fails
c
Just add it at the end, after your sink
@gentle-hamburger-31302 Can you give full example recipe ?
g
@alert-football-80212 Please find the sample recipe with transformers
Copy code
source:
  type: postgres
  config:
    # Coordinates
    host_port: localhost:5432
    database: postgres

    # Credentials
    username: postgres
    password: example

transformers:
  - type: "pattern_add_dataset_domain"
    config:
      domain_pattern:
        rules:
          'urn:li:dataset:\(urn:li:dataPlatform:postgres,postgres\.public\.n.*': ["urn:li:domain:test1"]
          'urn:li:dataset:\(urn:li:dataPlatform:postgres,postgres\.public\.t.*': ["test3"]
sink:
  type: "datahub-rest"
  config:
    server: "<http://localhost:8080>"
a
its worked. Thank you
what is the difference between - "urnlicorpuser:test_username" - "urnlicorpGroup:test_groupname"
c
a
Thank you!