https://linen.dev logo
#troubleshooting
Title
# troubleshooting
r

Rytis Zolubas

04/05/2022, 7:22 AM
S3 v0.3 destinaction deleted all my data in the bucket... in documentation I read why but still there should be never option to delete something after an upgrade. Pretty furious about this one
c

Chris Duong [Airbyte]

04/05/2022, 8:22 AM
Ouch 😱! So sorry to hear that…
in documentation I read why
Are you referring to?
⚠️ Please note that under “Full Refresh Sync” mode, data in the configured bucket and path will be wiped out before each sync. We recommend you to provision a dedicated S3 resource for this sync to prevent unexpected data deletion from misconfiguration. ⚠️
r

Rytis Zolubas

04/05/2022, 8:31 AM
@Chris Duong [Airbyte] correct. I had five connectors with incremental and one with full refresh... And everything was wiped out clean
c

Chris Duong [Airbyte]

04/05/2022, 8:36 AM
This behavior is not tied to the upgrade to v0.3 though, it’s been there since the beginning: https://github.com/airbytehq/airbyte/pull/3672/files#r641008420 but maybe we can work on improving on warning on the implications of the configuration better…
(or make the deletion revertable with some kind of trash area where the data is moved to for some time before being permanently deleted)
r

Rytis Zolubas

04/05/2022, 8:41 AM
before upgrading to v0.3 it would just truncate the files in the FOLDER now it deletes the entire BUCKET 😞
@Chris Duong [Airbyte] so I guess it is tied to the update
c

Chris Duong [Airbyte]

04/05/2022, 8:56 AM
I see, you’re right.
r

Rytis Zolubas

04/05/2022, 11:04 AM
would be great to solve this somehow, that it won't delete the entire bucket
c

Chris Duong [Airbyte]

04/05/2022, 11:05 AM
New version of the connector is published
r

Rytis Zolubas

04/05/2022, 6:43 PM
@Chris Duong [Airbyte] I noticed that it got merged, when it would be possible to see 0.3.1 version to download?
or is it merged to 0.3.0?
c

Chris Duong [Airbyte]

04/05/2022, 6:45 PM
0.3.1 you can try to upgrade your instance with it
r

Rytis Zolubas

04/05/2022, 7:36 PM
thanks a lot Chris!!
@Chris Duong [Airbyte] FYI s3 destination when selected to save as csv, did not add any file assignation in 0.3.0. In 0.3.1 it adds csv.gz
c

Chris Duong [Airbyte]

04/06/2022, 2:08 PM
yes i noticed that and fixed it while reworking the deletion
r

Rafael Auyer

04/08/2022, 1:05 AM
I'm late to the party but the same just happened to me 😢
c

Chris Duong [Airbyte]

04/08/2022, 8:48 AM
Sorry @Rafael Auyer about that… I did give you a (late?) warning on the other thread where you were asking to put
/
as your bucket path… I hope the damage was not too painful…
r

Rytis Zolubas

04/08/2022, 10:17 AM
@Rafael Auyer welcome to the party 😞
r

Rafael Auyer

04/08/2022, 2:40 PM
In my case I had a folder "airbyte", so damage was restricted to that folder.
4 Views