Airbyte is an open-source data integration engine that helps you consolidate your data in your data warehouses, lakes and databases.

Airbyte

Hi Team,
I was able to successfully set up airbyte on my local machine. Just curious when we deploy this to EC2 instance how can we setup user authentication for airbyte (self hosted )

Hi, we don't support authentication yet . The easiest way to do so would be to use an IAP on the dns Airbyte is exposed on on your internal network

What kind of user authentication are you looking for? Is it the ability to have different Airbyte workspaces per user? (different users will see different source/destinations/connections) Or is it securing the entire Airbyte deployment as a whole? (someone with the right password can access Airbyte)

Role Based Access Control (RBAC) would be ideal

Do you have any idea when you'll start working on authentication? We need to expose our instance to the internet to integrate with other components, and some people are understandably uncomfortable doing so without authentication

Would it address your use case to put nginx or another web server in front of airbyte, as a web proxy, and use authentication for that web server?  That way the airbyte instance would be on a private port, not exposed except by passing through nginx.

Hi <@U024VC21YRW>, the solution we came up with is to put an ALB in front of Airbyte and allow traffic only from specific IPs. Not quite as good as authentication (we cannot exclude that traffic we don't control could come from those IPs), but since we're only exposing two endpoints (trigger a sync and get the status of a job) it's good enough for now. We'd really like authentication though :simple_smile:

Your ALB could point at nginx and have nginx point at airbyte.  Use nginx for full authentication.  It would give you that while you wait for it to be a part of airbyte.

That could work, I'll bring it up next week. Appreciate your input.

One big flaw in our current setup (ALB &gt; Airbyte with no auth) is that `/api/v1/jobs/get` exposes secrets for some connectors, so in theory someone with access to the IPs we're whitelisting could trigger a sync, describe the job and get the credentials for the connectors used in the sync, if they're part of those which expose them

That’s entirely reasonable to be wary of, and is why putting a web server for auth in front of it is important.  It should also be using SSL for that web server, so you don’t risk known users viewing a page that shows secrets and having it end up going across the network in plain text.  Your ALB and web server proxy should keep https going for best safety.

You could also use something like <https://aws.amazon.com/blogs/aws/built-in-authentication-in-alb/|https://aws.amazon.com/blogs/aws/built-in-authentication-in-alb/> with the ALB and not need to configure an additional Nginx. Added benefit is it plugs into existing IDPs so no need to manage Nginx creds