Liren, I didn't realise we were using the S3 clien...
# contributing-to-airbyted
Davin Chia (Airbyte)
09/17/2021, 1:33 AM
Liren, I didn't realise we were using the S3 client under the hood for the GCS destination. Is that because the blob apis are actually compatible?
u
user
09/17/2021, 2:23 AMYes. Since S3 is so successful, it becomes kind of a protocol.
GCS and DigitalOcean both support it.
By reusing the S3 client, we was able to reuse all the S3 writers and support all four formats on GCS. Otherwise, Avro and Parquet formats can be tricky.
2 Views