Hey everybody, I’m planning on replacing a custom ...
# advice-data-ingestion
p
Hey everybody, I’m planning on replacing a custom SQS (to S3) ingestion script with Airbyte and I’m not sure if understand Airbyte’s approach correctly. I’m testing with a SQS queue where I’m publishing 1 message/second to. When I start Airbyte’s SQS sync job it seems to run until I quit my SQS-publishing script so Airbyte won’t find any more events. Our prod stage won’t (hopefully) be all empty for a second. Now the question: Is there any message or time limit until a SQS messages batch gets uploaded? I didn’t find any config or documentation for that but I may have missed something. Thanks in advance! 🙏