Hi, is there any way to use S3 as a source for datahub, but with my own endpoint that is not present on AWS?

Can you elaborate it a bit more what you exactly want to achieve?

So let's say that I have an S3 bucket on some non AWS server, what I would like to achieve is to make datahub connect to my S3, fetch the files and treat the jsons present there as a source. Right now as I am aware datahub supports only AWS S3 buckets, is there a possibility to define my own endpoint point to my internal S3 bucket?

Do you want to set the

endpoint_url

? It is currently not expose and unfortunately can’t be set for the s3 source, even though I think it wouldn’t be hard to add.

For the data source I would like to use the data-lake, and having the possibility to change endpoint_url is essential to make it work. For now we want to try experimental S3 source as well, which as far as I understood is the same as data-lake source but specific for S3. would there be possibility to make change to alter the endpoint_url?

Hello all. @dazzling-judge-80093 I also try to change data source from mysql to mssql. I deployed DataHub using docker container. How could I change endpoint url?