There seems to be an issue with the transformation...
# help-connector-development
j
There seems to be an issue with the transformations or the documentation on here https://docs.airbyte.com/connector-development/config-based/understanding-the-yaml-file/record-selector. I am trying to use it to remove unscary fields/arrays from a data souce, but i can't get it to work. Airbyte version (updated today): 0.42.0 (although been having the issue on 41 as well) CDK Version (again updated today but on previous version to): 0.30.1 sample from yaml
Copy code
record_selector:
        type: RecordSelector
        extractor:
          type: DpathExtractor
          field_path:
            - items
        transformations:
            - type: RemoveFields
              field_pointers:
                - [ "items", "links" ]
My json record response
Copy code
{
  "userId": 860,
  "firstName": "__WebServiceUser_",
  "email": "",
  "deleted": true,
  "mentionName": "@__webserviceuser_",
  "createdAt": "2021-08-24T00:02:04Z",
  "updatedAt": "2021-08-24T00:02:04Z",
  "links": {
    "self": "<https://eu2api.jobadder.com/v2/users/860>",
    "userGroups": "<https://eu2api.jobadder.com/v2/users/860/usergroups>"
  }
}
This should have removed the links array, but hasn't. I have tried with various field_pointers all with the same results. I also can't get the AddField transformation to work either.
a
transformations are applied on the extracted data. Try
Copy code
transformations:
            - type: RemoveFields
              field_pointers:
                - [ "links" ]
j
Hey @Alexandre Girard (Airbyte), tried that as well, same results. Also tried it without the record selector to remove a top level etc and still no luck.