Hi, I'm seeing some strange behavior with transfor...
# troubleshoot
a
Hi, I'm seeing some strange behavior with transformers. Please see recipe in thread. Version: 0.9.0 • If I try to add all 5 other transformers at one time, it DOES NOT WORK • If I add it by itself, it WORKS • If I add it first, by itself, then add the other 4 transformers after that and process again, it WORKS • If I add all 5 transformers to start, then remove them one-by-one, with just two other transformers remaining, it WORKS
Copy code
source:
   type: s3
   config:
       profiling:
           enabled: false
       path_specs:
           -
               include: '<s3://dev/Study2/Combined/*.*>'
       env: PROD
       aws_config:
           aws_access_key_id: '${AWS_ACCESS_KEY_ID_JR}'
           aws_secret_access_key: '${AWS_SECRET_KEY_JR}'
           aws_session_token: '${AWS_SESSION_TOKEN_JR}'
           aws_region: us-east-1
pipeline_name: 'urn:li:dataHubIngestionSource:61dcc24b-824c-4b60-858b-bd309e51c81a'
transformers:
   -
       type: simple_add_dataset_properties
       config:
           semantics: OVERWRITE
           properties:
               new_prop_1: Text
               new_prop_2: texttt
   -
       type: simple_add_dataset_tags
       config:
           tag_urns:
               - 'urn:li:tag:NeedsDocumentation'
               - 'urn:li:tag:Legacy'
   -
       type: simple_add_dataset_terms
       config:
           term_urns:
               - 'urn:li:glossaryTerm:Confidential'
   -
       type: simple_add_dataset_domain
       config:
           semantics: OVERWRITE
           replace_existing: true
           domains:
               - 'Molecular Analysis'
   -
       type: simple_add_dataset_ownership
       config:
           owner_urns:
               - 'urn:li:corpuser:accc8zz'
               - 'urn:li:corpGroup:Admin'
           ownership_type: PRODUCER
e
@gray-shoe-75895 have you seen anything like this before?
a
I created this github issue as directed by Pedro Silva: https://github.com/datahub-project/datahub/issues/6277