i'm using localstack for development rather than aws services — has anyone attempted to source files from localstack's s3 rather than The Real S3 in airbyte (for instance by changing the endpoint url or using importing localstack as boto3)

a quick search of

<site:docs.airbyte.io> localstack

on google yielded no results, and searching

localstack

in the repo on gh didn't either. figured i'd ask here before making any attempts

I don't think so. I'd be curious to find out what happens 🙂

We don’t have that configurable at the moment but we will need to allow configuration in the future. We were thinking about MinIO support as well, which would require something similar.

Would you mind creating an issue for allowing configurable S3 endpoints on Github?

@Jared Rhizor (Airbyte) can do 🙂 I was just digging through source-file and trying to see if it was as simple as importing localstack as boto3 or such… alas, it is not

Yeah it’ll be a bit tricky unfortunately.