Hello, so I am trying to ingest from Hive using so...
# troubleshoot
m
Hello, so I am trying to ingest from Hive using some transformers (in order to save me time from doing the respective additions by hand) and I think one of the transformers used is interfering in correct ingestion of the metadata. As you can see in the pictures the datasets ingested are like <db>.<table> when they should be <table>. Furthermore, no aditional metadata has been ingested of each dataset (like the info presented in properties) The transformers used are the following:
Copy code
transformers:
  -
     type: "pattern_add_dataset_tags"
     config:
       tag_pattern:
         rules:
           '.*030902.*': ['urn:li:tag:030902']
           '.*050501.*': ['urn:li:tag:050501']
  -
     type: "pattern_add_dataset_domain"
     config:
       domain_pattern:
         rules:
           '.*libros.*': ['urn:li:domain:c4c94633-96cf-4a93-baa7-15562905f8f0']
           '.*050501.*': ['urn:li:domain:97faf5f3-4494-4620-abcf-a6a9eeea9fbe']
I am using 0.9.0 for both gms and front and 0.0.8 for actions Both the tags and domains do exist and the execution was executed successfully. I think the problem could be in the
pattern_add_dataset_tags
(although I am not really sure) as I have previously used the latter transformer in previous ocasions and didn't have any problem with it. Thanks in advance!
b
hey Pablo! so if you ingest without using those transformers, the names of these datasets appear as you would expect? and do you see the tags and domains you would expect from these transformers?
m
Yes, without these transformers the names of the datasets are correct. No, I don't seem them, although I don't see either any error saying that they couldn't be added to the datasets