hi everyone, I intend to use Airbyte to load some data in csv from customer into own DB in Postgresql. The customer will upload the file in daily so I need to append data day by day by schedule daily also. I had read about SFTP or Google directory source but both not support incremental mode, Is the any source that support to append new data from CSV file ? or anyone have any solution for it?

I use the S3 connector, GCS is compatible with S3 so it can be used on GCP also.

Let me know if Marcelo's suggestion helps!

thanks @Marcelo Pio de Castro, I knew that S3 connector support incremental but from the customer they just only provide us via SFTP or Google drive folder

Hmm, you could use BigQuery to connect to your Google drive, and the use BigQuery as a source and send to Postgres

Hi everyone, I’m using Airbyte with S3 connector to load data from csv in S3 into Postgresql, but I faced problem that whenever the csv columns is not match the schema specified in connector to the exception occurred when we sync data. Does Airbyte support to validate csv before sync or not? or how can we prevent this case?

Hi Kelvin, I don't believe Airbyte validates CSVs before sync. For the future - please make sure to create a new thread if you have a new question!