Hello, we are just getting started with Airbyte, we have some data loads going between databases and Redshift and those are great. We are running into some problems though in loading csv files from SFTP sources. The normalization process is failing on them as it is casting the fields to the wrong data types automatically. I understand that the theory behind Airbyte is really focused on the EL layers and not so much transformation, but I’m trying to figure out the easiest way to modify the field types. Is the only way within Airbyte to build a dbt transformation that the connection can use, or are their any options that are easier to just change a few fields that are automatically detected as an incorrect type? Thanks for any help you can provide.

Maybe worth to take a look at this thread: https://airbytehq.slack.com/archives/C021JANJ6TY/p1625128615042500?thread_ts=1625098194.040200&cid=C021JANJ6TY

Thanks @Chris (deprecated profile) Forcing all columns to parse as strings would get us to at least get the data into fields, we would still have to deal with it again. I was pretty sure it was going to be worth our time to learn dbt transformations, I guess we better go that route because we’ll need to do that or something like it eventually.

What are the types that are not properly handled?