Hi everyone, what's the maximum file size that Airbyte can successfully process as a source?

The maximum file size that Airbyte can process depends on the specific feature and connector: - For the "raw file replication" feature (copying files without parsing), Airbyte supports files up to 1GB per file. This is available for certain connectors (e.g., SFTP Bulk to S3) and requires Airbyte version 1.2.0 or later. This limit is explicitly documented for file-based transfers, where files are copied as-is without parsing their contents [Copy Raw Files Configuration](https://docs.airbyte.com/integrations/enterprise-connectors/source-sharepoint-enterprise#copy-raw-files-configuration). - For standard record-based syncs (where data is parsed and processed as records), Airbyte Cloud has a maximum record size of 20MB. However, the effective maximum may be lower depending on the destination's constraints. For example, MongoDB has a 16MB record limit. If a record exceeds the destination's limit, Airbyte will attempt to adjust the record so the sync does not fail, retaining primary keys and cursors and storing modifications in

airbyte_meta.changes

for review [Airbyte Cloud limits](https://docs.airbyte.com/platform/cloud/managing-airbyte-cloud/understand-airbyte-cloud-limits). If you are referring to processing files as records (e.g., CSV, JSON), the 20MB per record limit applies. If you are using the raw file transfer feature, the 1GB per file limit applies. If you need to process larger files, you may need to split them before ingestion. If you have a specific connector or use case in mind, please provide more details for a more tailored answer.

i am reading Parquet file from S3. it is 12mb file. but it fails after 9 min but when i upload same type of file with less records, it works fine. what will be the issue, if 20MB is OK?