https://linen.dev logo
Channels
Powered by
l

Lukas Novotny

01/21/2022, 11:24 AM
Hello Airbyte team, we're testing your product in Kubernetes and it's working great! However, we are concerned about the unencrypted connection config storage in Postgres 
actor
table. Are we missing some trivial config that switches to a different secret store? Can we load secrets from environment variables?
j

Jens

01/21/2022, 1:29 PM
Hi @Lukas Novotny if you're on GCP you can use GSM to store secrets, this can be set through env vars, check this doc and let me know if it helps.
Unfortunately we're not on GCP, do we have other options?
a

Anouar Hnini

01/21/2022, 3:45 PM
Not at this time. We have a long standing issue to address this https://github.com/airbytehq/airbyte/issues/837
Hey @[DEPRECATED] Augustin Lafanechere - I enabled GSM by adding an entry in the the .env file at 
/kube/overlays/stable/.env
like so
Copy code
SECRET_PERSISTENCE_TYPE=GOOGLE_SECRET_MANAGER
And I verified by looking at the ConfigMap in GKE that the value is set. However no secrets seem to be created in my GSM and the server logs are still saying
Copy code
2022-01-23 15:35:52 [32mINFO[m i.a.c.EnvConfigs(getEnvOrDefault):596 - Using default value for environment variable SECRET_PERSISTENCE: 'NONE'
2022-01-23 15:35:52 [32mINFO[m i.a.c.EnvConfigs(getEnvOrDefault):596 - Using default value for environment variable SECRET_PERSISTENCE: 'NONE'
2022-01-23 15:35:52 [32mINFO[m i.a.c.EnvConfigs(getEnvOrDefault):596 - Using default value for environment variable SECRET_PERSISTENCE: 'NONE'
So looks like it’s not picking up the env var correctly? Or do I need set all of the variables defined in the doc you linked? I currently just set the 
SECRET_PERSISTENCE_TYPE
Edit: I tried setting all of them but still getting the same result
In fact seems that no matter what I put in the 
.env
file, the server logs will still show Airbyte choosing the default value (even when the ConfigMap reflects the value in the .env
Btw found that above is due to these env vars not being defined in the kubernetes manifests. I added them there and now I can override the defaults.
y

Yashasvi Chaudhary

01/24/2022, 4:00 PM
That said though… still seems theres issues getting the GCP service account credentials to format correctly… Do you have an example of how to configure e.g. the 
SECRET_STORE_GCP_CREDENTIALS
in the .secrets kustomize env file? @[DEPRECATED] Augustin Lafanechere
From the looks of it…. it seems that the secrets are stored in GSM as json objects, but then Airbyte fails to parse them as it expects a string. So this might be a bug tbh. Recreate: 1. Use GSM 2. Try to set up new BigQuery source pasting JSON credentials in the Airbyte UI 3. These credentials are stored in GSM as JSON objects 4. Airbyte tries to read them when checking connection with the source (and fails to parse them as it expects a string)
I think there is a bug 🐛 . The documentation at https://docs.airbyte.com/operator-guides/configuring-airbyte#secrets instruct to set a secret 
SECRET_PERSISTENCE_TYPE
however, the code expects 
SECRET_PERSISTENCE
https://github.com/airbytehq/airbyte/blob/5a091b12460ec6773c5655e3cae63248ad5cea86[…]e-config/models/src/main/java/io/airbyte/config/EnvConfigs.java
Yeah that is also a bug. Forgot to mention that 😅 But that I already worked around by setting 
SECRET_PERSISTENCE
instead. So even working around that bug, the above still happens which I think is a separate bug
e

Eugene Krall

01/25/2022, 10:02 AM
@[DEPRECATED] Augustin Lafanechere See above for 1 (possibly 2) bugs regarding the google secret manager implementation
e

Elias Djurfeldt

01/26/2022, 9:58 PM
Hey @Elias Djurfeldt thank you very much for this investigation! I'll open the issue you mention on our repo tomorrow, feel free to open them yourself.
Just to summarize, here are the three problems you raised: • GSM related env vars are not declared in Kube manifests • SECRET_PERSISTENCE_TYPE in documentation should be SECRET_PERSISTENCE • Deserialization of GSM secrets fails during connection check
The third issue is also a problem of documentation (specifying the format for the secrets stored in the env variable)
j

Jens

01/27/2022, 9:34 AM
Great - thank you @[DEPRECATED] Augustin Lafanechere! Yes as @Jared Rhizor (Airbyte) says, the third issue would also benefit from some additional documentation on expected format of the secrets. I tried a bunch of different formats and ways of specifying the service account before realising that there’s probably a bug in the deserialization.
d

Daniele

01/27/2022, 6:23 PM
@Elias Djurfeldt, I created two new issues: • 🐛 Secrets store: secrets related env vars are not declared in Kubernetes manifests • 🐛 Secret store: Deserialization of GSM secrets fails during connection check : Please share the Airbyte version you are using on this one Feel free to subscribe to these to receive updates from our team, and share the details of your investigations if needed. I opened a PR to fix the env var naming in the documentation.
p

Ping-Lin Chang

01/28/2022, 10:08 AM
Awesome - thank you! The bugs were found in 
0.35.6-alpha
Powered by