Hi everyone, is gzip format supported for s3 sourc...
# replication-troubleshooting
c
Hi everyone, is gzip format supported for s3 source? I don't see it in the source_definition_specifications for
S3 Source Spec
n
Hi Chen! We have extensive docs for each conenctor, please take a look. This file format is supported: https://docs.airbyte.com/integrations/sources/s3#file-compressions
c
@Nataly Merezhuk (Airbyte) thanks! I saw that documentation but I'm wondering is there any specific parameter I need to set when creating the s3 source? For now it says supported filetype are csv/parquet/json/avro, is gzip considered to be a separate filetype or it will just be gziped csv/parquet/json/avro files?
n
I see! I know in the S3 destination, gzip compression is supported for CSV and Jsonl. I'd need to dig into the connector code to see what's up with the s3 source. I can get back to you later on that or feel free to take a look yourself!
c
@Nataly Merezhuk (Airbyte) it'd be great if you can help take a look, I am specifically interested about s3 source support for gzip format, not the destination, thank you!
n
I understand! I'm looking at the code and don't see compression explicitly mentioned anywhere in the docs or in the code. I'll keep looking though!