Hi! I have been experimenting with Airbyte for the past few days to see if it can replace the python scripts I have written. I am very impressed with the tool, and I am all in on the open source community driven mission, but I think I need to wait a little while as there are some blockers to adoption in our case. 1. Changing table selections in the ui causes a full re sync of data. This is a pain, because on connectors like intercom/marketo, the initial sync can take 8+ hours. 2. Not being able to de select or hash specific columns is a concern for us because PII and other sensitive data in our analytics database means one more database with security / audit concerns. 3. Pre-defined schemas means that we will miss a lot of data as new columns are added to custom fields in apps like SFDC or Zendesk. It would be nice where possible to be able to opt out of column selection rather than opt in. 4. The salesforce connector doesn't support custom objects, which is a blocker. 5. The zoom connector is missing the phone/call endpoints 6. Connectors for SalesLoft / SaasOptics None of these issue on their own would stop me from adopting the tool, but all of them together severely limit the scope for which we can adopt it.

I think there are open issues covering most if not all of these, cruise by and throw a thumbs up on them! I have similar pain with the syncing all columns, and changing tables to cause resync

Thanks for the feedback! Like Jeff said, it would be helpful if you thumbs up/commented or left comments on existing issues or created new issues with your concerns 🙂

Custom dbt transformations. You can apply your own normalisations on the raw json records (via your own dbt project) and thus handle PII, predefined schemas, custom objects etc.

I will add these to github when I get some time later today. @gunu using dbt to hash PII or remove fields doesn't help in this context because the security implication is at the ingestion point. The fact that it is sitting in our database at all causes an issue and we need it to enter hashed, not hash after arrival. I am with you on the custom normalization for schemas but I think I wasn't clear in #3. If our SFDC admin adds a new column in salesforce, it doesn't get automatically pulled through to airbyte. I have to resync the whole connection to get that data. Its not a problem of normalization once the data gets there, its a problem of not getting it at all.