https://linen.dev logo
Join Slack
Powered by
# ask-ai
  • s

    Surendra Longre

    10/29/2025, 6:09 PM
    @kapa.ai Saved offset is before replication slot's confirmed lsn. Please reset the connection, and then increase WAL retention and/or increase sync frequency to prevent this from happening in the future. See https://docs.airbyte.com/integrations/sources/postgres/postgres-troubleshooting#under-cdc-incremental-mode-there-are-still-full-refresh-syncs for more details. Why I'm getting this error? I've the sync frequency of every 1 hour. then how come this error shows up?
    k
    • 2
    • 4
  • r

    Renu Fulmali

    10/29/2025, 6:18 PM
    @kapa.ai why theWORKLOAD_API_BEARER_TOKEN used?
    k
    • 2
    • 4
  • k

    Keegan Haukaas

    10/29/2025, 6:38 PM
    @kapa.ai What is the best way to add a second query parameter that is required along with a cursor when using Cursor Pagination in the Connector Builder?
    k
    • 2
    • 4
  • a

    Aviad Deri

    10/29/2025, 7:43 PM
    @kapa.ai trying to deploy airbyte 2.0 on kuberntes with external db but getting: TEMPORAL_ADDRESS is not set, setting it to 0.0.0.0:7233 PostgreSQL started. Setup PostgreSQL schema. 2025-10-29T193608.255Z ERROR Unable to create SQL database. {"error": "pq: permission denied to create database", "logging-call-at": "/home/runner/work/docker-builds/docker-builds/temporal/tools/sql/handler.go:93"}
    k
    • 2
    • 7
  • z

    Zack Mattor

    10/30/2025, 1:34 AM
    @kapa.ai I'm seeing the following logs in my airbyte worker pod...
    Copy code
    2025-10-30 01:19:43,623 [Workflow Executor taskQueue="ui_commands", namespace="default": 2]    WARN    i.t.i.r.ReplayWorkflowTaskHandler(failureToWFTResult):302 - Workflow task processing failure. startedEventId=95, WorkflowId=check_11cd23ea-7ca8-49c3-b319-a2ff805d7ddc, RunId=019a32ac-7b5f-7d84-a94b-40f368ab3786. If seen continuously the workflow might be stuck.
    io.temporal.internal.statemachines.InternalWorkflowTaskException: Failure handling event 4 of type 'EVENT_TYPE_WORKFLOW_TASK_COMPLETED' during replay. {WorkflowTaskStartedEventId=95, CurrentStartedEventId=3}
    Caused by: io.temporal.internal.sync.PotentialDeadlockException: [TMPRL1101] Potential deadlock detected. Workflow thread "workflow-method-check_11cd23ea-7ca8-49c3-b319-a2ff805d7ddc-019a32ac-7b5f-7d84-a94b-40f368ab3786" didn't yield control for over a second. {detectionTimestamp=1761787183417, threadDumpTimestamp=1761787183417} 
    
    workflow-method-check_11cd23ea-7ca8-49c3-b319-a2ff805d7ddc-019a32ac-7b5f-7d84-a94b-40f368ab3786
        at java.base/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base/java.util.concurrent.locks.LockSupport.park(LockSupport.java:221)
    k
    • 2
    • 7
  • m

    Marcos Takahashi

    10/30/2025, 2:11 AM
    @kapa.ai Does S3 Data Lake connector support S3 Table Buckets?
    k
    • 2
    • 4
  • z

    Zack Mattor

    10/30/2025, 2:34 AM
    @kapa.ai anything to note here?
    Copy code
    % helm upgrade airbyte airbyte-v2/airbyte \
      --namespace airbyte \
      --values ./values.yaml \
      --version 2.0.19
    
    I1029 22:33:23.241661   70396 warnings.go:110] "Warning: unknown field \"spec.template.spec.containers[0].allowPrivilegeEscalation\""
    I1029 22:33:23.241752   70396 warnings.go:110] "Warning: unknown field \"spec.template.spec.containers[0].capabilities\""
    I1029 22:33:23.241780   70396 warnings.go:110] "Warning: unknown field \"spec.template.spec.containers[0].fsGroup\""
    I1029 22:33:23.241797   70396 warnings.go:110] "Warning: unknown field \"spec.template.spec.containers[0].limits\""
    I1029 22:33:23.241829   70396 warnings.go:110] "Warning: unknown field \"spec.template.spec.containers[0].readOnlyRootFilesystem\""
    I1029 22:33:23.241849   70396 warnings.go:110] "Warning: unknown field \"spec.template.spec.containers[0].requests\""
    I1029 22:33:23.241871   70396 warnings.go:110] "Warning: unknown field \"spec.template.spec.containers[0].runAsGroup\""
    I1029 22:33:23.241887   70396 warnings.go:110] "Warning: unknown field \"spec.template.spec.containers[0].runAsNonRoot\""
    I1029 22:33:23.241901   70396 warnings.go:110] "Warning: unknown field \"spec.template.spec.containers[0].runAsUser\""
    I1029 22:33:23.241916   70396 warnings.go:110] "Warning: unknown field \"spec.template.spec.containers[0].seccompProfile\""
    Release "airbyte" has been upgraded. Happy Helming!
    NAME: airbyte
    LAST DEPLOYED: Wed Oct 29 22:32:49 2025
    NAMESPACE: airbyte
    STATUS: deployed
    REVISION: 4
    NOTES:
    Get the application URL by running these commands:
    k
    • 2
    • 1
  • i

    Ishan Anilbhai Koradiya

    10/30/2025, 4:59 AM
    @kapa.ai getting this error on the airbyte platform Warning from airbyte_platform: An internal transient Airbyte error has occurred. The sync should work fine on the next retry. Setting attempt to FAILED because the workflow for this connection was restarted, and existing job state was cleaned.
    k
    • 2
    • 1
  • h

    Horia

    10/30/2025, 4:59 AM
    @kapa.ai I ran into this problem while syncing MongoDB -> AWS RedShift
    Copy code
    io.airbyte.cdk.integrations.source.relationaldb.state.FailedRecordIteratorException: java.lang.RuntimeException: java.lang.RuntimeException: org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
    k
    • 2
    • 7
  • v

    Vajeen Karunathilaka

    10/30/2025, 8:36 AM
    @kapa.ai How can I contribute to airbyte helm chart
    k
    • 2
    • 2
  • a

    Aviad Deri

    10/30/2025, 9:19 AM
    @kapa.ai i'm tyring to install the ne airbyte version with the following values file postgresql: enabled: false global: database: type: external # -- Secret name where database credentials are stored secretName: "airbyte-config-secrets" # e.g. "airbyte-config-secrets" # -- The database host host: "10.10.10.10" # -- The database port port: "5432" # -- The database name database: "airbyteqa" # -- The database user userSecretKey: "database-user" # e.g. "database-user" # -- The key within
    secretName
    where password is stored passwordSecretKey: "database-password" # e.g."database-password" temporal: extraEnv: - name: SKIP_DB_CREATE value: "true" - name: DBNAME value: temporal_qa - name: VISIBILITY_DBNAME value: temporal_visibility_qa - name: POSTGRES_TLS_ENABLED value: "false" - name: SQL_TLS_ENABLED value: "false" by running the command abctl local install --secret secrets.yaml --values values.yaml but it failing with: ERROR i.m.m.h.i.HealthResult$Builder(exception):123 - Health indicator [10.10.10.10:5432/db-airbyte] reported exception: java.sql.SQLTransientConnectionException: HikariPool-2 - Connection is not available, request timed out after 30001ms (total=0, active=0, idle=0, waiting=0) for some reason it seems that the installation ignore the db name i gave in the config (airbyteqa) and trying to access db called db-airbyte. what is the problem?
    k
    • 2
    • 28
  • m

    Matthias Grawinkel

    10/30/2025, 9:37 AM
    @kapa.ai how to configure airbyte 2 helm chart to use a google cloud sql database via the cloud sql proxy with private ip addresses?
    k
    • 2
    • 2
  • s

    sp33dy

    10/30/2025, 10:01 AM
    @kapa.ai I have Airbyte deployed with helm chart v2 and I'm getting kyverno errors about missing or wrong securityContext in workload pods. How can I add it?
    k
    • 2
    • 1
  • d

    DR

    10/30/2025, 10:20 AM
    @kapa.ai I received a notice about a pending upgrade for the Facebook Marketing → Snowflake destination (v4.0.0) with breaking changes. In my setup, Airbyte automatically creates raw tables in the
    airbyte_internal
    schema (e.g.,
    _airbyte_raw_*
    ), and I only use the normalized tables in the
    facebook_marketing
    schema — I never directly query the raw tables. Given this, can I safely upgrade the destination directly without enabling the “Disable Final Tables” option? Or is there any recommended precaution I should take before upgrading
    k
    • 2
    • 1
  • m

    Mayur Shingare

    10/30/2025, 10:58 AM
    @kapa.ai how to get access of airbyte embedded
    k
    • 2
    • 7
  • s

    sp33dy

    10/30/2025, 11:04 AM
    @kapa.ai how can I contribute to your Helm charts?
    k
    z
    v
    • 4
    • 26
  • b

    Bryan Meyerovich

    10/30/2025, 1:25 PM
    i deployed on an ec2 using abctl https://docs.airbyte.com/platform/deploying-airbyte/abctl-ec2 i need it to have this code line changed https://github.com/airbytehq/airbyte/pull/69095 what steps do i need to do?
    k
    • 2
    • 1
  • b

    Bryan Meyerovich

    10/30/2025, 2:19 PM
    i want to upload an airbyte destination connector as a docker image. i want to make a version of s3 data lake destination connector with a single line of code modified
    k
    • 2
    • 8
  • a

    Aviad Deri

    10/30/2025, 2:26 PM
    @kapa.ai my syncs fails with "broken pipe". this is what i see in the log: 2025-10-30 160033 source WARN main c.n.s.UnknownKeywordFactory(lambda$getKeyword$0):37 Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword or if it should generate annotations AnnotationKeyword 2025-10-30 160033 source WARN main c.n.s.UnknownKeywordFactory(lambda$getKeyword$0):37 Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword or if it should generate annotations AnnotationKeyword 2025-10-30 160033 source WARN main c.n.s.UnknownKeywordFactory(lambda$getKeyword$0):37 Unknown keyword multiline - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword or if it should generate annotations AnnotationKeyword 2025-10-30 160033 source WARN main c.n.s.UnknownKeywordFactory(lambda$getKeyword$0):37 Unknown keyword display_type - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword or if it should generate annotations AnnotationKeyword 2025-10-30 160033 source WARN main c.n.s.UnknownKeywordFactory(lambda$getKeyword$0):37 Unknown keyword min - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword or if it should generate annotations AnnotationKeyword 2025-10-30 160033 source WARN main c.n.s.UnknownKeywordFactory(lambda$getKeyword$0):37 Unknown keyword max - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword or if it should generate annotations AnnotationKeyword 2025-10-30 160033 source WARN main c.n.s.UnknownKeywordFactory(lambda$getKeyword$0):37 Unknown keyword always_show - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword or if it should generate annotations AnnotationKeyword 2025-10-30 160117 destination WARN main c.k.j.j.JsonSchemaGenerator$MyJsonFormatVisitorWrapper(expectAnyFormat):725 Not able to generate jsonSchema-info for type: [simple type, class com.fasterxml.jackson.databind.JsonNode] - probably using custom serializer which does not override acceptJsonFormatVisitor 2025-10-30 160211 destination ERROR SLF4J(W): Class path contains multiple SLF4J providers. 2025-10-30 160211 destination ERROR SLF4J(W): Found provider [org.apache.logging.slf4j.SLF4JServiceProvider@2de8284b] 2025-10-30 160211 destination ERROR SLF4J(W): Found provider [org.slf4j.reload4j.Reload4jServiceProvider@396e2f39] 2025-10-30 160211 destination ERROR SLF4J(W): See https://www.slf4j.org/codes.html#multiple_bindings for an explanation. 2025-10-30 160211 destination ERROR SLF4J(I): Actual provider is of type [org.apache.logging.slf4j.SLF4JServiceProvider@2de8284b] 2025-10-30 160212 source WARN main c.n.s.UnknownKeywordFactory(lambda$getKeyword$0):37 Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword or if it should generate annotations AnnotationKeyword 2025-10-30 160212 source WARN main c.n.s.UnknownKeywordFactory(lambda$getKeyword$0):37 Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword or if it should generate annotations AnnotationKeyword 2025-10-30 160212 source WARN main c.n.s.UnknownKeywordFactory(lambda$getKeyword$0):37 Unknown keyword multiline - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword or if it should generate annotations AnnotationKeyword 2025-10-30 160212 source WARN main c.n.s.UnknownKeywordFactory(lambda$getKeyword$0):37 Unknown keyword display_type - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword or if it should generate annotations AnnotationKeyword 2025-10-30 160212 source WARN main c.n.s.UnknownKeywordFactory(lambda$getKeyword$0):37 Unknown keyword min - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword or if it should generate annotations AnnotationKeyword 2025-10-30 160212 source WARN main c.n.s.UnknownKeywordFactory(lambda$getKeyword$0):37 Unknown keyword max - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword or if it should generate annotations AnnotationKeyword 2025-10-30 160212 source WARN main c.n.s.UnknownKeywordFactory(lambda$getKeyword$0):37 Unknown keyword always_show - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword or if it should generate annotations AnnotationKeyword 2025-10-30 160216 destination WARN main c.k.j.j.JsonSchemaGenerator$MyJsonFormatVisitorWrapper(expectAnyFormat):725 Not able to generate jsonSchema-info for type: [simple type, class com.fasterxml.jackson.databind.JsonNode] - probably using custom serializer which does not override acceptJsonFormatVisitor 2025-10-30 160232 replication-orchestrator ERROR DestinationWriter error: 2025-10-30 160232 replication-orchestrator ERROR DestinationWriter error: 2025-10-30 160232 replication-orchestrator WARN Attempted to close a destination which is already closed. 2025-10-30 160232 replication-orchestrator ERROR DestinationReader error: 2025-10-30 160232 replication-orchestrator ERROR SourceReader error: 2025-10-30 160232 replication-orchestrator ERROR runJobs failed; recording failure but continuing to finish. 2025-10-30 160332 replication-orchestrator WARN Failed to wait for exit value file /dest/exitCode.txt to be found. 2025-10-30 160332 replication-orchestrator ERROR Error closing resource io.airbyte.container.orchestrator.worker.io.LocalContainerAirbyteDestination@6e576b5a; recording failure but continuing. 2025-10-30 160432 replication-orchestrator WARN Failed to wait for exit value file /source/exitCode.txt to be found. 2025-10-30 160432 replication-orchestrator ERROR Error closing resource io.airbyte.container.orchestrator.worker.io.LocalContainerAirbyteSource@32016cf3; recording failure but continuing. what is the issue?
    k
    • 2
    • 4
  • k

    Kuntal Basu

    10/30/2025, 2:40 PM
    upgrading postgres from version 13 to 16 any script i need to run
    k
    • 2
    • 1
  • s

    Stefano Messina

    10/30/2025, 2:44 PM
    @kapa.ai I'm testing the newest version of the clickhouse connector, which I configured to use a specific database where to create the tables. However it looks like it's trying to execute CREATE DATABASE <custom namespace>. How can I avoid that and have the connector just create tables in the database I provide
    k
    • 2
    • 4
  • s

    sp33dy

    10/30/2025, 2:53 PM
    @kapa.ai I see in values of chart v2 option globa.jobs.kube.localVolume. What is it for?
    k
    • 2
    • 4
  • s

    sp33dy

    10/30/2025, 3:10 PM
    @kapa.ai Airbyte contains some cert bundles in /etc/pki/ca-trust/extracted/openssl, /etc/pki/ca-trust/extracted/pem and /etc/pki/ca-trust/extracted/java. What they are need for and why Airbyte is creating them during deploy?
    k
    • 2
    • 1
  • t

    Todd Matthews

    10/30/2025, 3:17 PM
    how do I setup an instance admin email without navigating to the airbyte UI?
    k
    • 2
    • 1
  • z

    Zack Mattor

    10/30/2025, 3:56 PM
    @kapa.ai should a new pod be created when running a destination or source connection check?
    k
    • 2
    • 13
  • p

    Phil Pierce

    10/30/2025, 3:56 PM
    Install fails after an uninstall: INFO No existing cluster found, cluster 'airbyte-abctl' will be created SUCCESS Port 8000 appears to be available ERROR Cluster 'airbyte-abctl' could not be created ERROR unable to create kind cluster: failed to init node with kubeadm: command "docker exec --privileged airbyte-abctl-control-plane kubeadm init --config=/kind/kubeadm.conf --skip-token-print --v=6" failed with error: exit status 1: I1030 154046.437460 174 initconfiguration.go:261] loading configuration from "/kind/kubeadm.conf" W1030 154046.438537 174 common.go:101] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta3" (kind: "ClusterConfiguration"). Please use 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version.
    k
    • 2
    • 1
  • t

    Todd Matthews

    10/30/2025, 8:08 PM
    getting a 409 error after changing temporal database to external dbs
    k
    • 2
    • 4
  • t

    Todd Matthews

    10/30/2025, 8:47 PM
    how do I disable the airbyte minio chart?
    k
    • 2
    • 1
  • b

    Bryan Meyerovich

    10/30/2025, 9:16 PM
    i want to change a line of code in airbyte-cdk/bulk/toolkits/load-iceberg-parquet/src/main/kotlin/io/airbyte/cdk/load/toolkits/iceberg/parquet/io/IcebergTableCleaner.kt then build a docker image and use that image as the source for a new destination connector in my airbyte instance that is running from the main branch. how do i do that?
    k
    • 2
    • 2
  • p

    Phil Pierce

    10/30/2025, 9:28 PM
    When upgrading from 1.7.7 to 1.8.5 with abctl, the ingress nginx config is passing in the old service name: set $service_name "airbyte-abctl-airbyte-webapp-svc"; How is this fixed?
    k
    • 2
    • 1