Airbyte is an open-source data integration engine that helps you consolidate your data in your data warehouses, lakes and databases.

Airbyte

Hey All, Im running airbyte in a K8s cluster on GCP. But im not really happy with the way i need to use a gcp service account to write the logs to the log bucket. I would rather create a K8s service account and give that access to the bucket. Is there someone who can help me with pinpointing where in Airbyte the logs are actually written to the GCS bucket?

Hi <@U01CQT00Q72>, I think that the logic is handled in this class: airbyte-config/models/src/main/java/io/airbyte/config/helpers/GcsLogs.java

I think it uses the default credentials as backup. So im trying to create a K8s service account with the correct permissions

In `DefaultGcsClientFactory` it's specifically loading the client from the config specified by the env variable, which has GCS service creds. Pretty sure code changes are needed to allow dropping back to K8s service account.

Part of the problem is that we use the presence of these variables to decide which logging to do (S3, Minio, or GCS).