I am currently working on testing airbyte internal...
# ask-ai
m
I am currently working on testing airbyte internally using the helm-chart. Firstly I tried with postgresql.enabled = true which starts a postgres container. It worked in that case and I was able to access the webapp by port-forwarding. Then I tried using a dedicated postgres instance as data source for airbyte. In that case, I am facing some issue due to which temporal pod is not able to come up. Here is the log: ___________________________________ Waiting for Temporal server to start... + tctl cluster health + grep SERVING Waiting for Temporal server to start... + echo 'Waiting for Temporal server to start...' + sleep 1 _Unable to start server. Error: sql schema version compatibility check failed: pq: no pg_hba.conf entry for host "10.6.103.216", user "airbyte_admin_user", database "temporal", no encryption_ ___________________________________ Here is my values.yaml content. _______________________________________________________ postgresql: enabled: false temporal: extraEnv: _- name: "SKIP_SCHEMA_SETUP"_ value: "true" _- name: "SQL_TLS"_ value: "true" _- name: "SQL_TLS_DISABLE_HOST_VERIFICATION"_ value: "true" externalDatabase: _host: {{ $airbyte_db_application_secret.data.pg_host }}_ _user: {{ $airbyte_db_application_secret.data.pg_user }}_ _password: {{ $airbyte_db_application_secret.data.pg_password }}_ _database: {{ $airbyte_db_application_secret.data.db }}_ _port: {{ $airbyte_db_application_secret.data.pg_port }}_ _jdbcUrl: "jdbcpostgresql//{{ $airbyte_db_application_secret.data.pg_host }}:{{ $airbyte_db_application_secret.data.pg_port }}/{{ $airbyte_db_application_secret.data.db }}?stringtype=unspecified"_ _____________________________________________________________ Not able to figure out what's the problem. Please advise.
k
A message from kapa.ai
New message text here
m
this is not what i am looking for
s
The error says that you need to add 10.6.103.216 to the allowlist in the postgres pg_hba.conf. Can you check to make sure it's added?
m
@Shantanu Tripathi - please check if this is the concern
s
Actually its a managed rds, i don't think we can add this. Also it was working with postgres container earlier where I didn't modify pg_hba.conf as well. So not sure if it that was the issue
For your context, I am trying to deploy aiirbyte using helm chart and trying to use aws rds as airbyte db.
@Abhishek Garg
s
Oh ok, with rds I've gotten this error when I had force ssl enabled. can you try adding ssl require arguments to the jdbc string in jdbcUrl? e.g.
?ssl=true&sslmode=require
Or if that doesn't work please share any authentication settings you have configured in the rds db
s
Tried but getting the same issue. Authentication with rds is purely based on username and password and ssl is optional.
m
@Sunny Hashmi (Airbyte) - Please help here
s
Hey @Mukesh Gawadia does this look similar to your issue? If so, can you add your configs and logs to the issue as well as any additional context?
s
@Sunny Hashmi I had already mentioned the config (yaml) and the error logs. Do you want something in specific?
s
Hey all, please check out this thread: https://github.com/airbytehq/airbyte/issues/9342#issuecomment-1346262970 There's a workaround mentioned, can you see if it helps?
m
@Sunny Hashmi (Airbyte) - Thanks for help .Its working now .But we do have some questions 1. Airbyte config we are storing in RDS . For details around what does it store and how much storage is required so that we can have clarity on it. 2. During cdc i am assuming some storage again will be used for replication so lets have a understanding on that as well so that it doesn't affect other workloads. cc @Shantanu Tripathi
s
Hey @Mukesh Gawadia, you can get the answers to these questions in office hours next week 🙂 wednesday and thursday are covering the airbyte-db
m
thanks @Sunny Hashmi (Airbyte)
@Sunny Hashmi (Airbyte) Can you please help here to configure Airbyte internal DB connection . Right now airbyte alone is sitting on 20-30 connections on this rds instance. temporal, temporal_visibility, Airbyte collectivily taking more then 30 connection . How we can restrict this max connection ? i was exploring below link but not able to see any configuration for max connection https://docs.airbyte.com/operator-guides/configuring-airbyte-db/ cc @Shantanu Tripathi
👍 1
@Sunny Hashmi (Airbyte) any update on last query
@Sunny Hashmi (Airbyte) @[DEPRECATED] Marcos Marx - As we discussed on call .Please find screenshot which indicate connection distribution
Please help us to tune this well
p
@Mukesh Gawadia hey, I am running into the same situation as you but with a slightly different error message. can you tell me or give the example yaml to set this running?🥲