@kapa.ai hey, when you check for the cdc lsn, are you looking for all entries larger then the last lsn number or for everything between the last lsn and the largest lsn ? The question is in regard to following problem, what if the last lsn airbyte knows about is not available in the source system anymore ?

@kapa.ai can you help me with: “jsonschema.exceptions.ValidationError: The manifest version 2.3.0 is greater than the airbyte-cdk package version (2.0.0). Your manifest may contain features that are not in the current CDK version.”

What are these two variables for? # Override these if you want to access the API with known credentials #instance-admin-client-id: # my-client-id #instance-admin-client-secret: # my-client-secret

Hi @kapa.ai, I would like to add proxy to Airbyte Open Source inside a Linux VM. I tried to add proxy to • Docker-compose.yaml file [not working] • /etc/environment [not working] • /etc/systemd/system/docker.service.d/http-proxy.conf [not working] • JAVA_OPTS variable [not working] • JAVA_TOOL_OPTIONS variable [not working] Can someone please advise me where I should add proxy for Airbyte to accept it? Why do I need proxy? I wanted to change Clickhouse version from

1.0.0

to

0.2.5

to support normalization. Without changing proxy If there is a way to achieve it also - I am okay with it. Please assist.

@kapa.ai can you please show me the template for this part of the yaml? postgresqlenabled truepostgresqlUsername: airbytepostgresqlPassword: airbytepostgresqlDatabase: db-airbyte

@kapa.ai Failure in source: State code: 08001; Message: Public Key Retrieval is not allowed

@kapa.ai hey, when you check for the cdc lsn, are you looking for all entries larger then the last lsn number or for everything between the last lsn and the largest lsn ? The question is in regard to following problem, what if the last lsn airbyte knows about is not available in the source system anymore ?

Any plans on deprecating or removing the python cdk? Many sources are shifting to the low-code cdk so it's concerning.

I have deleted the source, destination, connection throught the API getting status 204 but I still can see them from the UI

@kapa.ai hey, while doing the sync airbyte mentions the target lsn. what is it and how does it relate to start and stop lsn of a sql server database

The HTTP 504 error you're encountering typically indicates a timeout error. This could be due to a variety of reasons such as network issues, server overload, or configuration problems. One of the users in a [Github issue](https://github.com/airbytehq/airbyte/issues/42135) faced a similar problem while configuring File connectors in Airbyte Open Source. The issue was resolved by increasing the resource limits of Docker Desktop (8CPU and 12 Gi) and then running

abctl local uninstall

followed by

abctl local install

. This resolved the 504 issue when creating connectors. Here are the steps they followed: 1. Run

kubectl get po -n airbyte-abctl

2. Identify any pods with status pending and "check" in the name 3. Describe pod

kubectl describe po -n airbyte-abctl [pending-pod-name]

4. Confirm if there are logs showing insufficient resourcing If you're using Airbyte Cloud, you might need to allow inbound traffic from Airbyte IPs as mentioned in the [Airbyte documentation](https://docs.airbyte.com/integrations/sources/mysql#step-4-airbyte-cloud-only-allow-inbound-traffic-from-airbyte-ips). If the issue persists, it would be helpful to provide more details about your setup such as the Airbyte version, source name/version, destination name/version, and any relevant logs. This will help in diagnosing the problem more accurately.

@kapa.ai While setting up mysql source in airbyte oss, i am getting below error. Unknown error (http.504.vDgodfRhKo3NM48N27BaVd)

#C01AHCD885S With the new setup Airbyte workspace Id got changed however the Airbyte url still pointing to the old workspace id*.* below url giving the job ids from old workspace https://localhost:1000/api/v1/jobs/get

@kapa.ai I got an error "Page size is invalid" when running Google Ads source. Help me resovle this problem

can i change the airbyteUrl in the values.yaml without deleting and recreating the namespace?

Yes, you can change the

airbyteUrl

in the

values.yaml

file without deleting and recreating the namespace. You can do this by using the

helm upgrade

command which updates the existing deployment with the new values specified in the

values.yaml

file. Here is an example of how you can do this:

bash helm upgrade \ --namespace airbyte \ --values ./values.yaml \ --install airbyte-enterprise \ airbyte/airbyte

In this command, replace

./values.yaml

with the path to your

values.yaml

file. This command will apply the new configuration to your existing Airbyte deployment in the specified namespace. Please note that you should be careful when changing the

airbyteUrl

as it is used in various places in your configuration. For instance, one user reported an issue where removing the

global.airbyteUrl

from the

values.yaml

configuration and manually changing the value of the

INTERNAL_API_HOST

environment variable in the

airbyte-airbyte-env

config map to remove the

http://

prefix helped fix a sync job failure issue after an upgrade ([source](https://github.com/airbytehq/airbyte/issues/38854)). Sources: - [Customizing your Deployment](https://docs.airbyte.com/enterprise-setup/implementation-guide#customizing-your-deployment) - [Configure your Deployment](https://docs.airbyte.com/enterprise-setup/implementation-guide#step-2-configure-your-deployment) - [Github Issue 38854](https://github.com/airbytehq/airbyte/issues/38854)

@kapa.ai how do i set up nginx ingress to work with the local ip address?

#C01AHCD885S With the new setup Airbyte workspace Id got changed however the get jobs api https://localhost:1000/api/v1/jobs/get fetching the job ids from old workspace. Restarted the webapp and all airbyte components but issue not resolved.

#C01AHCD885S @kapa.ai Workspace API, Source definitions API fetching the latest details but get Job API fetching the job details from old workspace.

when i first create a destination after first install of airbyte there is an error in server: Caused by: org.postgresql.util.PSQLException: ERROR: relation "secrets" does not exist

@kapa.ai I am using AWS Glue to managed iceberg metadata. when creating a iceberg destination, which catalog I could use

i am developing a connector with python cdk. How to use the values of the fields defined in spec.yaml?

Hey @kapa.ai what are the strategies for scaling recommended if we dont want to go down the kubernetes route ?

@kapa.ai which database is used by airbyte to store the metadata for sources,destinations or connections and also how the configurations are stored by the interface?

This message was deleted.

@kapa.ai, I need to hire someone who knows airbyte. Which Slack channel should I ping?

how to define a connector spec field as a timestamp and show the date picker at the interface

How to use partitioning in a custom connector with python cdk

how to retry on 404 errors and ignore after 3 tries?

how to retry on 404 errors and ignore after 3 tries in python with airbyte cdk??