:wave: Does anyone have any documentation on how t...
# ask-community-for-troubleshooting
d
đź‘‹ Does anyone have any documentation on how to configure Airbyte via Helm/K8s to work with our own Temporal Cloud Server?
g
not sure if this has changed, but I don't think there is direct support for this in the helm chart. we patch the rendered yaml for airbyte to change the TEMPORAL_HOST environment variable
d
Interesting. I’ll make note of that environment variable. Did you set
temporal.enabled
to False on your Helm chart values?
g
there are environment variables for temporal cloud https://github.com/airbytehq/airbyte/blob/master/airbyte-config/config-models/src/main/java/io/airbyte/config/EnvConfigs.java#L121-L125 maybe you can set them in env settings
yes
but we are running temporal in kubernetes separately, not temporal cloud. I would try to set the environment variables above in the worker and server extra environment variables
d
Good stuff, thanks for the Github links, not sure I would have thought to look there. I’ll take this to my infra team and see what we can do
k
you can, to do so just disable temporal by setting
temporal.enabled: false
and then pass the env vars for temporal cloud using
env_vars or extraEnv
block
Vars that should be used can be viewed in the link that @Guy Feldman shared above
d
Perfect, and these environment variables need to be part of the webapp or server or worker env?
k
Oh, wait, my bad, let me rephrase on of my messages
👍 1
You need to disasble temporal in airbyte chart and then deploy it as a separate one with
global.deploymentMode
set to anything other that
oss
, so the default values won’t be rendered. After that you can specify your own env var configuration by using
env_vars or extraEnv block
(first one is simple kv list, second one is list with dicts that allows you to specify values from secrets and configmaps)
d
Got it, as far as which to pass the environment variables to, would it be the webapp, server or worker?
or all 3?
Also, not familiar with the
deploymentMode
. If it isn’t
oss
what are the value options?
k
If this var is set to
oss
it will render default env setup that uses configmap, if not it won’t do it
You need to pass temporal connection vars only to worker and server
👍 1
d
Cool thanks. I’m unfamiliar with what
deploymentMode
value would be, but can it be ANYTHING but oss?
Yep anything but oss
it’s used for simple if statement check
d
Cool, thanks! Huge help! Thanks @Kyryl Skobylko and @Guy Feldman
k
Right now I’m stuck with AC tests for helm charts, but when it’ll be done, I’ll probably rework the logic of how we can specify our own configuration(e.g Temporal host, db connection, etc.) Since the current one is not that convenient for most of the users I guess
g
yeah I had a PR at one point to add a externalTemporal block just like there is an externalDatabase block. I did not have time to follow up or finish it
k
Probably we can combine those to into
extarnalConfiiguration
block?
So it’ll be stored in 1 place
g
i think it's good to have them separate but don't see the harm
👌 1
cause there's also externalMinio
that might get messy in the templates
k
@Dusty Shapiro have your or your team succeeded on using cloud temporal with oss helm charts?
d
We haven’t started the migration process yet - still determining how to surface certs
but it’s a card we’ll play in near future - can keep you updated
octavia loves 1
k
I’ll describe another way of migrating to cloud later
n
Thanks so much for all the info, @Kyryl Skobylko!
d
@Kyryl Skobylko Attempting this process as we speak. Will update you
k
Are you going to use temporal cloud?
Oh, yes, my bad, sorry.
d
I started to but hit some snags related to the non “oss” deploymentMode causing environment variables to not be present. Out of curiousity, it keeps complaning about this DATABASE_PASSWORD missing…but I’m using an external postgres database. Is this needed?
Another one before I call it a night:
Copy code
Exception in thread "main" java.lang.IllegalArgumentException: 'AIRBYTE_VERSION' environment variable cannot be null
Why can’t I use the OSS deployment mode? If I don’t, none of those env-configmap values are non-existant.
k
Hmm, actually you can I guess, you can try to override only those env vars that are required for temporal cloud to work
But it’s a good case to actually review the deployment mode variable strategy.
As for Airbyte Version env var missing, let me ask devs about it.
d
ok - going to keep at it today
When I add the verison as bootloader environment variable, the CRON pod fails with this error
Copy code
Message: Could not resolve placeholder ${DATABASE_PASSWORD}
I’m going to try to use the configmap as it seems like there are too many env values that need to be present. When I change my deploymentMode to something other than
oss
, all those values are null
So I’m curios what happens if I remove that condition
k
Have you tried to leave deploymentMode as is(i.e
oss
) and then specify only variables required for setting up temporal cloud connection?
This should probably do the trick, so you’ll both keep you configuration from configmap + specify overrides for temporal cloud
d
I didn’t, but let me backup to there and give it a shot.
As long as I set
temporal.enabled
to false it shouldn’t try to read the TEMPORAL_HOST I assume
And to confirm, here are the overriding keys that I’m passing in to the worker and server extraEnvs: • TEMPORAL_CLOUD_CLIENT_KEY • TEMPORAL_CLOUD_CLIENT_CERT • TEMPORAL_CLOUD_NAMESPACE • TEMPORAL_CLOUD_HOST • TEMPORAL_CLOUD_ENABLED
k
you can set TEMPORAL_HOST to
""
i.e empty string and it should do the trick too
As for the
temporal.enabled
switch, good catch, let me work on that and put an PR that fixes this logic.
d
Should the TEMPORAL_HOST be an environment variable in the configmap or the worker and server too
k
hmm, I think configmap is enough(if you can overwrite it), else set it for worker and server
As far as I remember it has half-hardcoded values, so better to set it for server and worker just in case to be sure it’s empty
d
Got it - I think that worked, now I’m getting Temporal specific errors related to my certs
Although in the DB Container I’m seeing:
Copy code
airbyte-db-0 airbyte-db-container 2022-12-15 03:38:13.473 UTC [465] FATAL:  password authentication failed for user "airbyte"
airbyte-db-0 airbyte-db-container 2022-12-15 03:38:13.473 UTC [465] DETAIL:  Password does not match for user "airbyte".
Not sure what this container is used for if I have an external DB setup?
Copy code
postgresql:
  enabled: false
k
hmm, let me check the sources
d
I got it working - the final step was fixing the certs
So it looks like having the deployment mode as
oss
made it a bit easier because it applies all of those environment variables.
đź‘€ 1
n
So glad you got this working! Saving this thread for future reference 🙂
d
Thanks! Happy to help. Also, not sure if it’s neccesary but I made sure in the server and worker that the TEMPORAL_HOST env variable was set to “”. I can remove that and see if it breaks it.
n
I'd be curious to know if it does!
d
Deploying to our dev environment now with those overriding values removed.
Ah, forgot i changed the value in my
env-configmap.yaml
to
""
as well. I am using my forked version of the Helm chart because I needed an ExternalSecret deployed, so I’ll need to update and push that too.