Airbyte is an open-source data integration engine that helps you consolidate your data in your data warehouses, lakes and databases.

Airbyte

Hi, I’m looking for some docs on how to use oauth with airbyte. We intend to send links to our clients where they do oauth-login, and are redirected back to our website at the end of the flow. I see Airbyte’s API exposes a few endpoints to manage oauth, is there support for an end-to-end flow where the refresh token is stored safely and automatically, or do we need to build our own credential manager to handle that?

Hi <@U02U2ELRRTL> could you be a bit more specific about the context in which you'd like to use oauth? Is it for authenticating users to Airbyte our to a custom connector you're working on?

So we are working with clients, and would like to easily set up connections to their Ecom-platforms to fetch and analyze their data.

The way we’re doing it with Fivetran is we get an authentication URL for the connector, that we send to the customer. They click that link, which essentially takes them to the Ecom-platform Oauth login flow. After finishing the login, the callback is handled by Fivetran and a refresh-token stored in fivetran — which means we can refresh data ourselves using the customers’ auth.

So to achieve this in Airbyte we’re looking at hosting our own database for refresh-tokens, and build our own oauth url-generator and callback handler — but we realize too that to leverage this auth we’ll need to modify code in most Airbyte connectors so we’d be very keen to learn if there’s support for something similar in Airbyte already. Otherwise I’d be happy to discuss how we could contribute to it.

I believe this is the setup at ft: <https://fivetran.com/docs/getting-started/powered-by-fivetran>

<@U02U2ELRRTL> I think we only currently support this kind of flow on Airbyte Cloud. <@U02TFM12THU> do you think the <https://airbyte-public-api-docs.s3.us-east-2.amazonaws.com/rapidoc-api-docs.html#tag--oauth|Oauth endpoints> we expose in our API can support  Jens' usecase?

We’re looking to host ourselves, and are not afraid to build functionality nor contribute it upstream if relevant :slightly_smiling_face:

but wouldn’t want to reinvent something that’s already supported

Hey Jens, the oauth endpoints are used with Airbyte cloud and aren’t really meant for OSS usage at the moment (it’s certainly not tested in the context of OSS :smile:) but you are welcome to kick the tires on it!

Worst case you can always implement your own oauth flow and interact with oauth-based connectors via the API just like any other connector

Thanks <@U02TFM12THU> , I see some sources should be rather easy to patch up, like the `Shopify` one that has abstracted out the http oauth header generation to a separate function in an `auth.py` module (that we can shadow with our own method). For others like e.g. Chargebee it seems harder as the api-key based auth is baked into the Source-class’ `streams` method itself.

I guess what I’m getting at is, is there a best-practice or style guide we can adhere to, and perhaps a kind request to make the broken-out auth as in `Shopify` a part of the connector-template cookiecutter :slightly_smiling_face:

Hey <@U02U2ELRRTL> can you create an issue around it so that we can discuss and track it

<@U02U2ELRRTL> Did you ever create an issue for this? We are in a similar boat. We have our own OAuth implementation that our clients use to connect their accounts (in our system) to third-party services. We maintain their OAuth access tokens in our backend and would like to pass them to Airbyte for use in user-specific connections.