@kapa.ai I upgraded a postgres destination connector to version 3 form v2. Now in my warehouse i am facing the issue where ince the Dalgo Postgres Destination Connector upgrade, our pipeline and connectors have been failing. The cause is how the upgraded connector names columns. For long column names (mostly form question fields), it now cuts the name short and adds a number at the end. For example, in the CSAT_Form_Report table: Attach check and tell me

@kapa.ai what's the latest version for sql-server

@kapa.ai is there a patch planned for the eMudhra Private keys that were compromised

@kapa.ai getting this error when running the jira connector 2026-10-26 065032 replication-orchestrator INFO failures: [ { "failureOrigin" : "destination", "failureType" : "system_error", "internalMessage" : "tech.allegro.schema.json2avro.converter.AvroConversionException: Failed to convert JSON to Avro: Could not evaluate union, field lead is expected to be one of these: NULL, STRING. If this is a complex type, check if offending field (path: lead) adheres to schema: {self=saasxl.atlassian.net/rest/api/3/user?accountId=…, accountId=712020:e91d92a4-60fc-4e8d-b494-6348ad63a6c7, accountType=atlassian, avatarUrls={48x48=, 24x24=, 16x16=, 32x32=}, displayName=Jyotish, active=true}", "externalMessage" : "Something went wrong in the connector. See the logs for more details.", "metadata" : { "attemptNumber" : 4, "jobId" : 22459, "from_trace_message" : true, "connector_command" : "write" }, "stacktrace" : "tech.allegro.schema.json2avro.converter.AvroConversionException: Failed to convert JSON to Avro: Could not evaluate union, field lead is expected to be one of these: NULL, STRING. If this is a complex type, check if offending field (path: lead) adheres to schema: {self=saasxl.atlassian.net/rest/api/3/user?accountId=…, accountId=712020:e91d92a4-60fc-4e8d-b494-6348ad63a6c7, accountType=atlassian, avatarUrls={48x48=, 24x24=, 16x16=, 32x32=}, displayName=Jyotish, active=true}\n\tat tech.allegro.schema.json2avro.converter.JsonGenericRecordReader.read(JsonGenericRecordReader.java:131)\n\tat tech.allegro.schema.json2avro.converter.JsonGenericRecordReader.read(JsonGenericRecordReader.java:120)\n\tat tech.allegro.schema.json2avro.converter.JsonAvroConverter.convertToGenericDataRecord(JsonAvroConverter.java:95)\n\tat io.airbyte.cdk.integrations.destination.s3.avro.AvroRecordFactory.getAvroRecord(AvroRecordFactory.java:39)\n\tat io.airbyte.cdk.integrations.destination.s3.parquet.ParquetSerializedBuffer.accept(ParquetSerializedBuffer.java:97)\n\tat io.airbyte.cdk.integrations.destination.record_buffer.SerializedBufferingStrategy.addRecord(SerializedBufferingStrategy.java:73)\n\tat io.airbyte.cdk.integrations.destination.buffered_stream_consumer.BufferedStreamConsumer.acceptTracked(BufferedStreamConsumer.java:205)\n\tat io.airbyte.cdk.integrations.base.FailureTrackingAirbyteMessageConsumer.accept(FailureTrackingAirbyteMessageConsumer.java:65)\n\tat io.airbyte.cdk.integrations.base.Destination$ShimToSerializedAirbyteMessageConsumer.consumeMessage(Destination.java:115)\n\tat io.airbyte.cdk.integrations.base.Destination$ShimToSerializedAirbyteMessageConsumer.accept(Destination.java:91)\n\tat io.airbyte.cdk.integrations.base.IntegrationRunner.consumeWriteStream(IntegrationRunner.java:321)\n\tat io.airbyte.cdk.integrations.base.IntegrationRunner.consumeWriteStream(IntegrationRunner.java:301)\n\tat io.airbyte.cdk.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.java:190)\n\tat io.airbyte.cdk.integrations.base.IntegrationRunner.run(IntegrationRunner.java:125)\n\tat io.airbyte.integrations.destination.gcs.GcsDestination.main(GcsDestination.java:16)\n\tSuppressed: java.lang.NullPointerException: Cannot invoke \"java.io.InputStream.close()\" because \"this.inputStream\" is null\n\t\tat io.airbyte.cdk.integrations.destination.s3.parquet.ParquetSerializedBuffer.close(ParquetSerializedBuffer.java:163)\n\t\tat io.airbyte.cdk.integrations.destination.record_buffer.SerializedBufferingStrategy.close(SerializedBufferingStrategy.java:159)\n\t\tat io.airbyte.cdk.integrations.destination.buffered_stream_consumer.BufferedStreamConsumer.close(BufferedStreamConsumer.java:301)\n\t\tat io.airbyte.cdk.integrations.base.FailureTrackingAirbyteMessageConsumer.close(FailureTrackingAirbyteMessageConsumer.java:82)\n\t\tat io.airbyte.cdk.integrations.base.Destination$ShimToSerializedAirbyteMessageConsumer.close(Destination.java:96)\n\t\tat io.airbyte.cdk.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.java:189)\n\t\t... 2 more\nCaused by: org.apache.avro.AvroTypeException: Could not evaluate union, field lead is expected to be one of these: NULL, STRING. If this is a complex type, check if offending field (path: lead) adheres to schema: {self=saasxl.atlassian.net/rest/api/3/user?accountId=…, accountId=712020:e91d92a4-60fc-4e8d-b494-6348ad63a6c7, accountType=atlassian, avatarUrls={48x48=, 24x24=, 16x16=, 32x32=}, displayName=Jyotish, active=true}\n\tat tech.allegro.schema.json2avro.converter.AvroTypeExceptions.unionException(AvroTypeExceptions.java:16)\n\tat tech.allegro.schema.json2avro.converter.JsonGenericRecordReader.readUnion(JsonGenericRecordReader.java:304)\n\tat tech.allegro.schema.json2avro.converter.JsonGenericRecordReader.read(JsonGenericRecordReader.java:198)\n\tat tech.allegro.schema.json2avro.converter.JsonGenericRecordReader.read(JsonGenericRecordReader.java:172)\n\tat tech.allegro.schema.json2avro.converter.JsonGenericRecordReader.lambda$readRecord$0(JsonGenericRecordReader.java:154)\n\tat java.base/java.util.LinkedHashMap.forEach(LinkedHashMap.java:986)\n\tat tech.allegro.schema.json2avro.converter.JsonGenericRecordReader.readRecord(JsonGenericRecordReader.java:143)\n\tat tech.allegro.schema.json2avro.converter.JsonGenericRecordReader.read(JsonGenericRecordReader.java:129)\n\t... 14 more\n", "timestamp" : 1793011772220 }, { "failureOrigin" : "destination", "internalMessage" : "Destination process exited with non-zero exit code 1", "externalMessage" : "Something went wrong within the destination connector", "metadata" : { "attemptNumber" : 4, "jobId" : 22459, "connector_command" : "write" }failureOrigin" : "replication", "internalMessage" : "io.airbyte.workers.exception.WorkerException: Destination process exit with code 1. This warning is normal if the job was cancelled.", "externalMessage" : "Something went wrong during replication", "metadata" : { "attemptNumber" : 4, "jobId" : 22459 }, "stacktrace" : "java.lang.RuntimeException: io.airbyte.workers.exception.WorkerException: Destination process exit with code 1. This warning is normal if the job was cancelled.\n\tat io.airbyte.workers.general.BufferedReplicationWorker$CloseableWithTimeout.lambda$close$0(BufferedReplicationWorker.java:548)\n\tat io.airbyte.workers.general.BufferedReplicationWorker.lambda$runAsyncWithTimeout$5(BufferedReplicationWorker.java:244)\n\tat java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\nCaused by: io.airbyte.workers.exception.WorkerException: Destination process exit with code 1. This warning is normal if the job was cancelled.\n\tat io.airbyte.workers.internal.LocalContainerAirbyteDestination.close(LocalContainerAirbyteDestination.kt:61)\n\tat io.airbyte.workers.general.BufferedReplicationWorker$CloseableWithTimeout.lambda$close$0(BufferedReplicationWorker.java:546)\n\t... 5 more\n", "timestamp" : 1793011772506 } ] any issue in the source credential ?

2026-10-26 162948 platform > Retry State: RetryManager(completeFailureBackoffPolicy=BackoffPolicy(minInterval=PT10S, maxInterval=PT30M, base=3), partialFailureBackoffPolicy=null, successiveCompleteFailureLimit=5, totalCompleteFailureLimit=10, successivePartialFailureLimit=1000, totalPartialFailureLimit=10, successiveCompleteFailures=4, totalCompleteFailures=4, successivePartialFailures=0, totalPartialFailures=0) 2026-10-26 162948 platform > Backing off for: 4 minutes 30 seconds. 2026-10-26 163419 platform > Docker volume job log path: /tmp/workspace/17341/4/logs.log 2026-10-26 163419 platform > Executing worker wrapper. Airbyte version: 0.59.1 2026-10-26 163419 platform > Attempt 0 to save workflow id for cancellation 2026-10-26 163419 platform > 2026-10-26 163419 platform > ----- START CHECK ----- 2026-10-26 163419 platform > 2026-10-26 163419 platform > Using default value for environment variable SIDECAR_KUBE_CPU_LIMIT: '2.0' 2026-10-26 163419 platform > Using default value for environment variable SOCAT_KUBE_CPU_LIMIT: '2.0' 2026-10-26 163419 platform > Using default value for environment variable SIDECAR_KUBE_CPU_REQUEST: '0.1' 2026-10-26 163419 platform > Using default value for environment variable SOCAT_KUBE_CPU_REQUEST: '0.1' 2026-10-26 163419 platform > Checking if airbyte/source-declarative-manifest:0.85.0 exists... 2026-10-26 163419 platform > airbyte/source-declarative-manifest:0.85.0 not found locally. Attempting to pull the image... 2026-10-26 163419 platform > Image does not exist. 2026-10-26 163419 platform > Unexpected error while checking connection: io.airbyte.workers.exception.WorkerException: Could not find image: airbyte/source-declarative-manifest:0.85.0 at io.airbyte.workers.process.DockerProcessFactory.create(DockerProcessFactory.java:117) ~[io.airbyte-airbyte-commons-worker-0.59.1.jar:?] at io.airbyte.workers.process.AirbyteIntegrationLauncher.check(AirbyteIntegrationLauncher.java:149) ~[io.airbyte-airbyte-commons-worker-0.59.1.jar:?] at io.airbyte.workers.general.DefaultCheckConnectionWorker.run(DefaultCheckConnectionWorker.java:71) ~[io.airbyte-airbyte-commons-worker-0.59.1.jar:?] at io.airbyte.workers.general.DefaultCheckConnectionWorker.run(DefaultCheckConnectionWorker.java:44) ~[io.airbyte-airbyte-commons-worker-0.59.1.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.get(TemporalAttemptExecution.java:142) ~[io.airbyte-airbyte-workers-0.59.1.jar:?] at io.airbyte.workers.temporal.check.connection.CheckConnectionActivityImpl.lambda$runWithJobOutput$1(CheckConnectionActivityImpl.java:226) ~[io.airbyte-airbyte-workers-0.59.1.jar:?] at io.airbyte.commons.temporal.HeartbeatUtils.withBackgroundHeartbeat(HeartbeatUtils.java:57) ~[io.airbyte-airbyte-commons-temporal-core-0.59.1.jar:?] at io.airbyte.workers.temporal.check.connection.CheckConnectionActivityImpl.runWithJobOutput(CheckConnectionActivityImpl.java:211) ~[io.airbyte-airbyte-workers-0.59.1.jar:?] at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) ~[?:?] at java.base/java.lang.reflect.Method.invoke(Method.java:580) ~[?:?] at io.temporal.internal.activity.RootActivityInboundCallsInterceptor$POJOActivityInboundCallsInterceptor.executeActivity(RootActivityInboundCallsInterceptor.java:64) ~[temporal-sdk-1.22.3.jar:?] at io.temporal.internal.activity.RootActivityInboundCallsInterceptor.execute(RootActivityInboundCallsInterceptor.java:43) ~[temporal-sdk-1.22.3.jar:?] at io.temporal.internal.activity.ActivityTaskExecutors$BaseActivityTaskExecutor.execute(ActivityTaskExecutors.java:107) ~[temporal-sdk-1.22.3.jar:?] at io.temporal.internal.activity.ActivityTaskHandlerImpl.handle(ActivityTaskHandlerImpl.java:124) ~[temporal-sdk-1.22.3.jar:?] at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handleActivity(ActivityWorker.java:278) ~[temporal-sdk-1.22.3.jar:?] at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:243) ~[temporal-sdk-1.22.3.jar:?] at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:216) ~[temporal-sdk-1.22.3.jar:?] at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:105) ~[temporal-sdk-1.22.3.jar:?] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?] at java.base/java.lang.Thread.run(Thread.java:1583) [?:?] 2026-10-26 163419 platform > 2026-10-26 163419 platform > ----- END CHECK ----- 2026-10-26 163419 platform > 2026-10-26 163420 platform > Retry State: RetryManager(completeFailureBackoffPolicy=BackoffPolicy(minInterval=PT10S, maxInterval=PT30M, base=3), partialFailureBackoffPolicy=null, successiveCompleteFailureLimit=5, totalCompleteFailureLimit=10, successivePartialFailureLimit=1000, totalPartialFailureLimit=10, successiveCompleteFailures=5, totalCompleteFailures=5, successivePartialFailures=0, totalPartialFailures=0) Backoff before next attempt: 13 minutes 30 seconds 2026-10-26 163420 platform > Failing job: 17341, reason: Job failed after too many retries for connection 6ca944eb-9d82-42be-ad16-3093298c400b

@kapa.ai Are there any problems with having 2 connections write data to the same schema?

HI @kapa.ai, can I confirm whether it's possible to configure date partitioning of the airbyte logs (not the destination connectors, but the log messages from the syncs) in S3?

since last week, using

airbyte/source-e2e-test:2.2.2

is failing the sync due to net.jimblackler.jsonschemafriend.GenerationException: java.io.FileNotFoundException: raw.githubusercontent.com/fent/…/randexp.min.js - is this a known issue

how to refresh destination schema for a data activation connection? i manually call discover_schema endpoint with the destination id but it doesnt update it in the UI, there's no refresh schema option in the connection panel itself

@kapa.ai my amplitude connector keeps getting stuck. Any suggestions?

@kapa.ai i updated google sheet connector version in one of the airbyte workspaces, but it got updated in across all the sources and in all wokrspaces, why ?

@kapa.ai connection with the source

airbyte/source-mongodb-v2:2.0.7

is failing once in a while with "Terminating due to java.lang.OutOfMemoryError: Java heap space" error. Data volume of a single sync does not drift much. Full refresh always finishes successfully. What could be be an underlying issue?

is there any way to not have redshift-destination always write varchar(6565535) its bad for redshift query performance - why does it not mirror the length of the source where possible

@kapa.ai Can we add extra volumes/volumemounts to the dynamically created sync/check/etc pods via the helm chart?

@kapa.ai, how can I check what is the JDBC version used by airbyte?

@kapa.ai google analytics ga4 connector has consolidated streams , does they have incremental append

@kapa.ai I have some questions about the GA4 connector. Is there a way to get a unique ID associated with an individual user session or individual user? If so what stream does that live in?

This message was deleted.

Can a custom connector developed with CDK version ^7 can work on the Airbyte platform v 2.0.1 ?

after upgrade msserver version from 4v to 5v...it used to work cdcd at 4v

Incumbent CDC state is invalid, reason: io.airbyte.cdk.ConfigErrorException: CDC is not available for the following table(s)

@kapa.ai is there a fix for using

instanceprofile

for AWS secrets for storing connector secrets? Currently this does not work, and Airbyte only accepts

aws-secret-manager-access-key-id

and

aws-secret-manager-secret-access-key

keys. Doc: docs.airbyte.com/platform/deploying-airbyte/integrations/secrets

@kapa.ai I’m setting up an API stream in the custom builder. I want to use the partition router to iterate through a list of values from another stream. The key values need to be inserted into the endpoint URL. Is there a way to do this? For example purposes you can use id and event_id as the Parent Key and Current Parent Key Value Identifier

@kapa.ai a sync job with an MSSQL source is causing

tempdb

to fill, so the job ends with an exception:

io.airbyte.cdk.integrations.source.relationaldb.state.FailedRecordIteratorException: java.lang.RuntimeException: com.microsoft.sqlserver.jdbc.SQLServerException: An error occurred during the current command (Done status 0). Could not allocate space for object 'dbo.SORT temporary run storage:  164013481918464' in database 'tempdb' because the 'PRIMARY' filegroup is full due to lack of storage space or database files reaching the maximum allowed size. Note that UNLIMITED files are still limited to 16TB. Create the necessary space by dropping objects in the filegroup, adding additional files to the filegroup, or setting autogrowth on for existing files in the filegroup.

is there anything that can be tuned on the Airbyte side?

@kapa.ai how can i enable debug logging for replication pods to gather more information about failed or stuck syncs?

what causes occasional Failure in destination: Failed to initialize connector operation issues in postgres+iceberg connection?

2026-11-09 20:02:32 replication-orchestrator INFO Failures: [ {
  "failureOrigin" : "destination",
  "failureType" : "config_error",
  "internalMessage" : "io.airbyte.cdk.ConfigErrorException: Failed to initialize connector operation",
  "externalMessage" : "Failed to initialize connector operation",
  "metadata" : {
    "attemptNumber" : 0,
    "jobId" : 102256,
    "from_trace_message" : true,
    "connector_command" : "write"
  },
  ..................................
2026-11-09 20:02:33 platform INFO
----- START POST REPLICATION OPERATIONS -----

2026-11-09 20:02:33 platform INFO No post-replication operation(s) to perform.
2026-11-09 20:02:33 platform INFO
----- END POST REPLICATION OPERATIONS -----

2026-11-09 20:02:33 platform INFO Retry State: RetryManager(completeFailureBackoffPolicy=BackoffPolicy(minInterval=PT10S, maxInterval=PT30M, base=3), partialFailureBackoffPolicy=null, successiveCompleteFailureLimit=5, successivePartialFailureLimit=1000, totalCompleteFailureLimit=10, totalPartialFailureLimit=20, successiveCompleteFailures=1, successivePartialFailures=0, totalCompleteFailures=1, totalPartialFailures=0)
 Backoff before next attempt: 10 seconds
2026-11-09 20:02:33 platform ERROR Failing job: 102256, reason: io.airbyte.cdk.ConfigErrorException: Failed to initialize connector operation

What if I set “Initial Waiting Time in Seconds (Advanced)” to a value greator than 2400?

@kapa.ai hey bro, happy diwali. facing an issue with one of our custom no code connector. when trying to do a clear stream getting this error → 2026-11-11 173037 destination ERROR main i.a.c.AirbyteConnectorRunnable(run):38 Failed class io.airbyte.cdk.load.write.WriteOperation operation execution. java.lang.IllegalArgumentException: sourceStats must be set with recordCount for state message id=1 helpplease

@kapa.ai Hi! We manage Airbyte (OSS, Helm chart 1.8.5) with the Terraform provider airbytehq/airbyte 1.3.0 (also checked 1.3.2) and are trying to bring data activation connections under Terraform: source-redshift 0.5.5 -> destination-hubspot 0.0.12. Connections built in the UI use Full refresh + Upsert (stored as syncMode=full_refresh, destinationSyncMode=append_dedup, with matching key as primary key). The HubSpot destination catalog declares only append_dedup with matchingKeys for CONTACT, COMPANY and our custom object. Two problems via the public API / provider: 1. The connection sync_mode enum has no value for Full refresh + Upsert (there is full_refresh_update, full_refresh_soft_delete, incremental_deduped_history, but no full refresh + append_dedup). GET /api/public/v1/streams for this source/destination returns only [“full_refresh_append”, “incremental_append”]. Omitting sync_mode creates the connection with overwrite and drops the primary key. 2. Connections created via POST /api/public/v1/connections (with destination_object_name set) have no destination catalog; the UI Mappings tab shows “This connection has not been configured with a destination catalog.” Is there a supported way to create or update data activation connections with Full refresh + Upsert through the public API or the Terraform provider? Is this a known gap, or is it fixed in a newer Airbyte version?

@kapa.ai hello bro, another day another airbyte issue → so one of our connection is mongodb connection which has CDC enabled with resync data + capture mode as lookup. The issue is every night one of the table gets written and its size is around 64gb in the source. we don’t pull this table into the waehouse using airbyte. but still the airbyte logs show pulling op logs for around 45 mins and moving aournd 70 gb from our infra but in reality only 30 mb of data gets written to the wahouse. the table that we sync is migrants, and the catalog table name is migrants_catalog_green. Can you tell me why this might be happening ?

Hey @kapa.ai we’re using airbyte core, and getting Hmmm… can’t reach this page localhost refused to connect when trying to access airbyte. can you walk me through some troubleshooting steps? we’ve restarted the server aribyte is running on itself but i want to check docker etc