Airbyte is an open-source data integration engine that helps you consolidate your data in your data warehouses, lakes and databases.

Airbyte

Hi team, was curious about the S3 destination connector. Specifies `Please note that the stream name may contain a prefix, if it is configured on the connection. A data sync may create multiple files as the output files can be partitioned by size (targeting a size of 200MB compressed or lower) .`

Is there a way (either UI or code changes) to change the 200mb to a different chunking size?

the note in the readme changed <https://github.com/airbytehq/airbyte/pull/11496/files>, however i dont see anything directly pointing to differences there in the code (although im not the most familiar)

I found `MAX_PER_STREAM_BUFFER_SIZE_BYTES` in `FileBuffer.java`, which looks like it could be modified in the codebase if needed

That is just to split the upload into parts to avoid upload issues; it does not have any impact on the actual file size storage in S3. <https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpuoverview.html>