Hi

Currently, each data sync will only create one file per stream. In the future, the output file can be partitioned by size. Each partition is identifiable by the partition ID, which is always 0 for now.

Ref: https://docs.airbyte.io/integrations/destinations/s3 When will we get this feature? Parsing/reading from S3 files of size 10s of GBs would be complicated.

Answered in this thread: https://airbytehq.slack.com/archives/C01VDDEGL7M/p1632856656011900?thread_ts=1632851381.010700&cid=C01VDDEGL7M