S3 v0.3 destinaction deleted all my data in the bu...
# troubleshootingr
Rytis Zolubas
04/05/2022, 7:22 AMS3 v0.3 destinaction deleted all my data in the bucket... in documentation I read why but still there should be never option to delete something after an upgrade. Pretty furious about this one
c
Chris Duong [Airbyte]
04/05/2022, 8:22 AMOuch 😱!
So sorry to hear that…
in documentation I read whyAre you referring to?
⚠️ Please note that under “Full Refresh Sync” mode, data in the configured bucket and path will be wiped out before each sync. We recommend you to provision a dedicated S3 resource for this sync to prevent unexpected data deletion from misconfiguration. ⚠️
r
Rytis Zolubas
04/05/2022, 8:31 AM@Chris Duong [Airbyte] correct. I had five connectors with incremental and one with full refresh... And everything was wiped out clean
c
Chris Duong [Airbyte]
04/05/2022, 8:36 AMThis behavior is not tied to the upgrade to v0.3 though, it’s been there since the beginning: https://github.com/airbytehq/airbyte/pull/3672/files#r641008420
but maybe we can work on improving on warning on the implications of the configuration better…
Chris Duong [Airbyte]
04/05/2022, 8:37 AM(or make the deletion revertable with some kind of trash area where the data is moved to for some time before being permanently deleted)
r
Rytis Zolubas
04/05/2022, 8:41 AMbefore upgrading to v0.3 it would just truncate the files in the FOLDER
now it deletes the entire BUCKET 😞
Rytis Zolubas
04/05/2022, 8:45 AM@Chris Duong [Airbyte] so I guess it is tied to the update
c
Chris Duong [Airbyte]
04/05/2022, 8:56 AMI see, you’re right.
r
Rytis Zolubas
04/05/2022, 11:04 AMwould be great to solve this somehow, that it won't delete the entire bucket
c
Chris Duong [Airbyte]
04/05/2022, 11:05 AMChris Duong [Airbyte]
04/05/2022, 4:58 PMNew version of the connector is published
r
Rytis Zolubas
04/05/2022, 6:43 PM@Chris Duong [Airbyte] I noticed that it got merged, when it would be possible to see 0.3.1 version to download?
Rytis Zolubas
04/05/2022, 6:44 PMor is it merged to 0.3.0?
c
Chris Duong [Airbyte]
04/05/2022, 6:45 PM0.3.1 you can try to upgrade your instance with it
r
Rytis Zolubas
04/05/2022, 7:36 PMthanks a lot Chris!!
Rytis Zolubas
04/06/2022, 2:07 PM@Chris Duong [Airbyte] FYI s3 destination when selected to save as csv, did not add any file assignation in 0.3.0.
In 0.3.1 it adds csv.gz
c
Chris Duong [Airbyte]
04/06/2022, 2:08 PMyes i noticed that and fixed it while reworking the deletion
r
Rafael Auyer
04/08/2022, 1:05 AMI'm late to the party
but the same just happened to me 😢
c
Chris Duong [Airbyte]
04/08/2022, 8:48 AMSorry @Rafael Auyer about that…
I did give you a (late?) warning on the other thread where you were asking to put
/r
Rytis Zolubas
04/08/2022, 10:17 AM@Rafael Auyer welcome to the party 😞
r
Rafael Auyer
04/08/2022, 2:40 PMIn my case I had a folder "airbyte", so damage was restricted to that folder.
4 Views