@Nataly Merezhuk (Airbyte) thanks! I saw that documentation but I'm wondering is there any specific parameter I need to set when creating the s3 source? For now it says supported filetype are csv/parquet/json/avro, is gzip considered to be a separate filetype or it will just be gziped csv/parquet/json/avro files?
n
Nataly Merezhuk (Airbyte)
02/02/2023, 7:27 PM
I see! I know in the S3 destination, gzip compression is supported for CSV and Jsonl. I'd need to dig into the connector code to see what's up with the s3 source. I can get back to you later on that or feel free to take a look yourself!
c
Chen Lin
02/02/2023, 8:33 PM
@Nataly Merezhuk (Airbyte) it'd be great if you can help take a look, I am specifically interested about s3 source support for gzip format, not the destination, thank you!
n
Nataly Merezhuk (Airbyte)
02/06/2023, 1:34 PM
I understand! I'm looking at the code and don't see compression explicitly mentioned anywhere in the docs or in the code. I'll keep looking though!