@kapa.ai Is there a way to get Airbyte syncs metadata information such as status, last updated, etc to save in my database?

getting this error when running abctl local install WARNING Encountered an issue deploying Airbyte: Pod: airbyte-abctl-bootloader.18662a20371d2312 Reason: Failed Message: Error: couldn't find key instance-admin-password in Secret airbyte-abctl/airbyte-abctl-airbyte-secrets Count: 9 DEBUG Received unexpected event: *v1.Status DEBUG Event watcher completed. ERROR Failed to install airbyte/airbyte Helm Chart ERROR Unable to install Airbyte locally ERROR unable to install airbyte chart: unable to install helm: context canceled

how many workspace can i create on https://cloud.airbyte.com/ on free trail

@kapa.ai getting this failure message on 24 hour sync frequency: Saved offset no longer present on the server. Please reset the connection, and then increase binlog retention and/or increase sync frequency.

@kapa.ai My airbyte OSS is hosted on docker compose(on ec2 linux). I got this output: [ec2-user@ip-10-103-3-23 airbyte]$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES e5fe127587ad airbyte/worker:0.57.2 "/bin/bash -c airbyt…" 7 hours ago Up 7 hours 5005/tcp, 0.0.0.0:32776->9000/tcp, :::32776->9000/tcp airbyte-worker c079db9d6966 airbyte/db:0.57.2 "docker-entrypoint.s…" 8 hours ago Up 7 hours 5432/tcp airbyte-db 9183b98683c9 airbyte/temporal:0.57.2 "./update-and-start-…" 8 hours ago Up 7 hours 6933-6935/tcp, 6939/tcp, 7233-7235/tcp, 7239/tcp airbyte-temporal fc246d82a2d9 alpine/socat "socat -d -d -t 900 …" 8 hours ago Up 7 hours 0.0.0.0:32772->2375/tcp, :::32772->2375/tcp ec2-user-docker-proxy-1 010eb31a5d9c airbyte/proxy:0.62.0 "./run.sh ./run.sh" 9 hours ago Up 8 hours 0.0.0.0:8000-8001->8000-8001/tcp, :::8000-8001->8000-8001/tcp, 0.0.0.0:8003->8003/tcp, :::8003->8003/tcp, 80/tcp, 0.0.0.0:8006->8006/tcp, :::8006->8006/tcp airbyte-proxy-old 6b7983bd398d airbyte/worker:0.62.0 "/bin/bash -c airbyt…" 9 hours ago Up 8 hours 5005/tcp, 0.0.0.0:32769->9000/tcp, :::32769->9000/tcp airbyte-worker-old 9a2377426860 airbyte/server:0.62.0 "/bin/bash -c airbyt…" 9 hours ago Up 8 hours 5005/tcp, 8000/tcp, 0.0.0.0:32768->8001/tcp, :::32768->8001/tcp airbyte-server-old 6705606b6dae airbyte/cron:0.62.0 "/bin/bash -c airbyt…" 9 hours ago Up 8 hours airbyte-cron-old 832ff56fdd9c airbyte/connector-builder-server:0.62.0 "/bin/bash -c airbyt…" 9 hours ago Up 8 hours 0.0.0.0:32771->8080/tcp, :::32771->8080/tcp airbyte-connector-builder-server-old 78a527bb9541 airbyte/webapp:0.62.0 "/docker-entrypoint.…" 9 hours ago Up 8 hours 8080/tcp airbyte-webapp-old 1ddd082efefe airbyte/airbyte-api-server:0.62.0 "/bin/bash -c airbyt…" 9 hours ago Up 8 hours 5005/tcp, 0.0.0.0:32770->8006/tcp, :::32770->8006/tcp airbyte-api-server-old 363d2616aaea airbyte/db:0.62.0 "docker-entrypoint.s…" 9 hours ago Up 8 hours 5432/tcp airbyte-db-old-09172025 0303e033eab1 airbyte/temporal:0.62.0 "./update-and-start-…" 9 hours ago Up 8 hours 6933-6935/tcp, 6939/tcp, 7233-7235/tcp, 7239/tcp airbyte-temporal-old ed2e23846158 alpine/socat "socat -t 900 TCP-LI…" 9 hours ago Up 7 hours 0.0.0.0:32777->2375/tcp, :::32777->2375/tcp airbyte-docker-proxy-1 Then I did docker stop airbyte-worker airbyte-db airbyte-temporal ec2-user-docker-proxy-1 and after this airbyte localhold 8000 did not open. then I did docker compose down and if doing docker compose up -d it is giving repsonsesd like Error response from daemon: Conflict. The container name "/airbyte-worker" is already in use by container "e5fe127587adde2be151e34e1a87abad0455b3c89ccafa72bc2ad4f0c7191080". You have to remove (or rename) that container to be able to reuse that name. for different containers each time. Help me

@kapa.ai why is the freshdesk connector not throwing error when the credentials are invalid ?

how to manually trigger syncs through api

@kapa.ai error message getting triggered "The Apache Avro library failed to read data with the following error: This type does not have attribute"

@kapa.ai BigQuery connector setup failing with the message

com.google.cloud.bigquery.BigQueryException: Connect timed out

@kapa.ai i am syncing data from jira to snowflake but i am getting below error message='Airbyte could not track the sync progress. Sync process exited without reporting status.', type='io.airbyte.workers.exception.WorkloadMonitorException', nonRetryable=false

@kapa.ai Why MySQL destination DB name limit is 44 symbols?

@kapa.ai in the ui there is an extracted rows and loaded rows per job. But I dont' see this in the API reference. Where can I see when there was a difference in extracted and loaded?

We are currently self hosting Airbyte on Kubernetes, we would like to understand what would be required worker configuration if we were to switch to self hosting Enterprise. Is there a way to calculate the required workers for enterprise licensing ?

I have created a pipeline from mongo to clickhouse and its incremental refresh and in append and dedup mode but its duplicating records in destination

Hi @kapa.ai , I’m running Airbyte via

abctl

in a single-container VM (

kindest/node:v1.32.2

). The server is healthy:

curl <http://localhost:8000/api/v1/health>
{"available":true}

But when I call the discover schema endpoint for a valid source ID:

curl -v -X POST <http://localhost:8000/api/v1/sources/discover_schema> \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer <API_TOKEN>" \
  -d '{"sourceId": "<source-id>"}'

I get a 502 Bad Gateway HTML response from Airbyte’s internal NGINX:

<h1>An error occurred.</h1>
<p>Sorry, the page you are looking for is currently unavailable.</p>
<p><em>Faithfully yours, nginx.</em></p>

•

sourceId

is valid, headers and token are correct. •

/api/v1/health

works. • The container is running and stable. Is this a known issue with

discover_schema

in single-container abctl setups, or is there a missing configuration I should check? Thanks!

hi @kapa.ai 0 bytes|no records loaded|31m 11s sync is finished but no data

@kapa.ai what is this error Activity with activityType='AttemptFailureWithAttemptNumber' failed: 'Activity task timed out'. scheduledEventId=174, startedEventId=175, activityId=49999a63-6d54-3308-8830-4f566e8d8674, identity='', retryState=RETRY_STATE_MAXIMUM_ATTEMPTS_REACHED

@kapa.ai can I obtain from internal airbyte postgres the list of source and target tables?

@kapa.ai can I use incremental deduped mode on my airbyte OSS syncs to Mongodb Atlas ?

@kapa.ai connection from googleads to clickhouse fails even in the first sync for the account_performance_report stream with this error Code: 524. DB:Exception ALTER of key column customer_id from type Int64 to type Nullable(Int64) is not safe because it can change the representation of primary key. (ALTER_OF_COLUMN_IS_FORBIDDEN) (version 25.6.2.6054 (official build))

@kapa.ai We’ve been experiencing 409 errors from the source-google-sheets connector when updating schemas

@kapa.ai I have created a mysql as source with cdc and required permissions enabled. My destination is clickhouse. I have created a connection using "incremental | append + deduped". It captures inserts but it creates duplicated of updates which is not expected. It should instead just update the row at the destination without creating a new one. Also, tried this using both ways CDC(source defined cursor) and with custom defined cursor. Issue is with how data is pulled into the detination DB and stored. Give me a resolution and ask if you have any follow up questions.

@kapa.ai hey there, I've been trying to install airbyte on my local machine for a while now, it seems it takes a lot of time and not much is happening, did anybody encounter this as well? My internet connection is also stable and should be fast enough. I have been waiting for approx 45 minutes. The errors I receive are: ERROR Failed to install airbyte/airbyte Helm Chart ERROR Unable to install Airbyte locally ERROR unable to install airbyte chart: unable to install helm: context canceled

@kapa.ai I have deployed airbyte on GCP kube cluster. Seeing error when accessing builder endpoint: you may need to update your ingress rules. I have specified the following in my ingress config:

spec:
  ingressClassName: nginx
  rules:
    - host: <http://host.example.com|host.example.com>
      http:
        paths:
          - backend:
              service:
                # format is ${RELEASE_NAME}-airbyte-connector-builder-server-svc
                name: airbyte-airbyte-connector-builder-server-svc
                port:
                  number: 80 # service port, example: 8080
            path: /api/v1/connector_builder/
            pathType: Prefix
          - backend:
              service:
                # format is ${RELEASE_NAME}-airbyte-server-svc
                name: airbyte-airbyte-server-svc
                port:
                  number: 8001 # service port, example: 8080
            path: /
            pathType: Prefix

@kapa.ai We setup syncs from SQL server to Snowflake and the initial sync works. However incremental syncs do not transfer any records where they should have been transferred. Logs do not show any errors

@kapa.ai My Airbyte pipelines started hanging hence I did docker compose down and up due to which airbyte instance is upgraded to 0.62 version. After upgrade, if I am clicking any pipeline to sync now, nothing is happening. My source and Destinations are not passing test connections. What can I do to make it working?

@kapa.ai how to setup the RDS as external database for airbyte

@kapa.ai Unexpected Exception java.lang.NullPointerException: Cannot invoke "io.airbyte.api.model.generated.AirbyteCatalog.getStreams()" because "discovered" is null

#C01AHCD885S will airbyte 2.0 have CDC for the Oracle DB connector? Why does the Oracle DB connector not have cdc?

#C01AHCD885S why does the open-source oracle connector not have CDC?