Hello all! We are exploring Airbyte for our data integration use cases and a question came up. In our existing ingest, we have the need to exclude some columns from a database source (Postgres). As an example, we would like to ingest user records but exclude PII in the table. Is that possible to do with existing connectors or would we need to create a custom one?
09/22/2021, 6:00 PM
Without needing anything fancy in the connector, you can put a view on the table, used specifically for data transfer, and eliminate those columns via the view.