@kapa.ai one more question about difference between using S3 and Minio, I noticed that using S3 our sync is slower, might be due to network, but where we host minio closely to airbyte self-hosted instance, the speed is higher.

Your observation makes sense from a networking perspective. When MinIO is hosted close to (or on the same infrastructure as) your self-hosted Airbyte instance, the latency between Airbyte and the storage backend is much lower compared to reaching out to AWS S3 over the public internet or a remote network. However, the knowledge sources don't explicitly discuss or compare the sync speed differences between using S3 vs. MinIO as log/state storage backends in terms of performance benchmarks. What the sources do note is: • MinIO is the default internal storage for self-hosted Airbyte deployments, and it's co-located with the Airbyte instance by default. [Configuring Airbyte] • For Self-Managed Enterprise, Airbyte actually recommends moving away from the default internal MinIO to external storage like S3 or GCS for reliability reasons, not necessarily for speed. [Enterprise Implementation Guide] • Airbyte has made general sync speed improvements (4–10x faster) for certain connector combinations, but these are related to connector-level optimizations, not storage backend choice. [Faster sync speed] So while your hypothesis about network proximity being the cause of the speed difference is reasonable, the knowledge sources don't contain enough information to confirm or quantify the performance difference between S3 and MinIO as storage backends in Airbyte specifically. If latency to S3 is a concern, you might consider using an S3 bucket in the same AWS region as your Airbyte infrastructure to minimize that overhead. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.