Hi all, relatively new to Airbyte here. I’m curren...
# replication-troubleshooting
j
Hi all, relatively new to Airbyte here. I’m currently ingesting a google sheet to Snowflake. We added a new column to the sheet and I believe that the connector will not update the schema during a regular sync--at least mine didn’t. I tried a reset as well, but it still doesn’t update the schema in Snowflake. I explored the UI in the connection as well as the source connector, and there doesn’t seem to be any way to update the schema. The only thing that worked was deleting the source connector entirely and recreating it. I saw during the initial setup that it’s “querying the schema” and so I believe that’s the only place/time that the schema can be changed. The problem here is that I can’t delete and recreate the source connector every time a column is added on the google sheet, particularly because I need to capture it in historical/append mode. It seems that Airbyte wants to reset the stream anytime I do anything, like changing sync modes, so I’m pretty sure I can’t just recreate the source connector and point it to previously gathered data. That’s my assumption at least. If I do that, I’ll lose the history. Either way, I’m hoping to confirm the following: The google sheet connector in fact does not allow for schema changes during regular syncs, without resetting data. Or am I overlooking something here? Thanks!
e
Hi. you are correct this can be done only by using airbyte API. I am not sure as to the best way to do it since you need to update the source (which you can using the API) but if you have CI|CD this could complicate things as this can result in building the image again... and again... for each update
💯 1
j
Perfect, that’s what I was hoping to confirm. I’ll look into the API, but at least in the short term I have another workaround. Thanks again!
airbyte rocket 1
n
Glad to hear this was resolved, thanks Eli!
octavia partying 1