Hey everybody, I’m planning on replacing a custom SQS (to S3) ingestion script with Airbyte and I’m not sure if understand Airbyte’s approach correctly.
I’m testing with a SQS queue where I’m publishing 1 message/second to.
When I start Airbyte’s SQS sync job it seems to run until I quit my SQS-publishing script so Airbyte won’t find any more events. Our prod stage won’t (hopefully) be all empty for a second.
Now the question: Is there any message or time limit until a SQS messages batch gets uploaded? I didn’t find any config or documentation for that but I may have missed something.
Thanks in advance! 🙏