@kapa.ai What does this error point to in a destination postgres sync: invalid input syntax for type double precision: "None"

@kapa.ai Analyze the following logs

@kapa.ai whcih is the best way to deploy airbyte . My company is trying to ingest data aroound 1tb per day from 20 different source databases on a daily basis ,,if i want to deploy airbyte on aws we are thinking of deploying it on a large ec2 instance ..is it best if we do it on eks or can we deploy it on ec2 and is it best if we install it using helm charts on k3s or using abctl and which underlyingos is best windows or liunux ...explain me why i should take the particular option clearly

@kapa.ai If the page contains less than 5 records, then the paginator knows there are no more pages to fetch. If the API returns more records than requested, all records will be processed. The paginator stops by default if the number of records is less than the max page size - but it stops wrongly in some cases. For example, I have an API that returns max 200 records, it does return 200 for page 1 and 2, then returns 199 (for whatever reason) which stops airbyte, even though there is still a page 3 , 4 , 5 etc after. Is there a way around this?

@kapa.ai trying to set potsgres source on Kubernetes installation but getting the following 2025-11-25 162827 INFO 2025-11-25 162827 info 2025-11-25 162827 INFO 2025-11-25 162827 info Transitioning workload to running state 2025-11-25 162827 INFO 2025-11-25 162827 info ----- START CHECK ----- 2025-11-25 162827 INFO 2025-11-25 162827 info 2025-11-25 162827 INFO 2025-11-25 162827 info Workload successfully transitioned to running state 2025-11-25 162848 INFO 2025-11-25 162848 info Connector exited, processing output 2025-11-25 162848 INFO 2025-11-25 162848 info Output file jobOutput.json found 2025-11-25 162848 INFO 2025-11-25 162848 info Connector exited with exit code 0 2025-11-25 162848 INFO 2025-11-25 162848 info Reading messages from protocol version 0.2.0 2025-11-25 162848 INFO 2025-11-25 162848 info INFO main i.a.i.s.p.PostgresSource(main):750 starting source: class io.airbyte.integrations.source.postgres.PostgresSource 2025-11-25 162848 INFO 2025-11-25 162848 info INFO main i.a.c.i.b.IntegrationCliParser$Companion(parseOptions):144 integration args: {check=null, config=/config/connectionConfiguration.json} 2025-11-25 162848 INFO 2025-11-25 162848 info INFO main i.a.c.i.b.IntegrationRunner(runInternal):130 Running integration: io.airbyte.cdk.integrations.base.ssh.SshWrappedSource 2025-11-25 162848 INFO 2025-11-25 162848 info INFO main i.a.c.i.b.IntegrationRunner(runInternal):131 Command: CHECK 2025-11-25 162848 INFO 2025-11-25 162848 info INFO main i.a.c.i.b.IntegrationRunner(runInternal):132 Integration config: IntegrationConfig{command=CHECK, configPath='/config/connectionConfiguration.json', catalogPath='null', statePath='null'} 2025-11-25 162848 WARN 2025-11-25 162848 warn WARN main c.n.s.UnknownKeywordFactory(lambda$getKeyword$0):37 Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword or if it should generate annotations AnnotationKeyword 2025-11-25 162848 WARN 2025-11-25 162848 warn WARN main c.n.s.UnknownKeywordFactory(lambda$getKeyword$0):37 Unknown keyword group - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword or if it should generate annotations AnnotationKeyword 2025-11-25 162848 WARN 2025-11-25 162848 warn WARN main c.n.s.UnknownKeywordFactory(lambda$getKeyword$0):37 Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword or if it should generate annotations AnnotationKeyword 2025-11-25 162848 WARN 2025-11-25 162848 warn WARN main c.n.s.UnknownKeywordFactory(lambda$getKeyword$0):37 Unknown keyword always_show - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword or if it should generate annotations AnnotationKeyword 2025-11-25 162848 WARN 2025-11-25 162848 warn WARN main c.n.s.UnknownKeywordFactory(lambda$getKeyword$0):37 Unknown keyword pattern_descriptor - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword or if it should generate annotations AnnotationKeyword 2025-11-25 162848 WARN 2025-11-25 162848 warn WARN main c.n.s.UnknownKeywordFactory(lambda$getKeyword$0):37 Unknown keyword multiline - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword or if it should generate annotations AnnotationKeyword 2025-11-25 162848 WARN 2025-11-25 162848 warn WARN main c.n.s.UnknownKeywordFactory(lambda$getKeyword$0):37 Unknown keyword display_type - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword or if it should generate annotations AnnotationKeyword 2025-11-25 162848 WARN 2025-11-25 162848 warn WARN main c.n.s.UnknownKeywordFactory(lambda$getKeyword$0):37 Unknown keyword min - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword or if it should generate annotations AnnotationKeyword 2025-11-25 162848 WARN 2025-11-25 162848 warn WARN main c.n.s.UnknownKeywordFactory(lambda$getKeyword$0):37 Unknown keyword max - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword or if it should generate annotations AnnotationKeyword 2025-11-25 162848 WARN 2025-11-25 162848 warn WARN main c.n.s.UnknownKeywordFactory(lambda$getKeyword$0):37 Unknown keyword groups - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword or if it should generate annotations AnnotationKeyword 2025-11-25 162848 INFO 2025-11-25 162848 info INFO main i.a.c.i.b.s.SshTunnel$Companion(getInstance):424 Starting connection with method: NO_TUNNEL 2025-11-25 162848 INFO 2025-11-25 162848 info INFO main i.a.i.s.p.PostgresUtils(isCdc):70 using CDC: false 2025-11-25 162848 INFO 2025-11-25 162848 info INFO main i.a.i.s.p.PostgresSource(toSslJdbcParamInternal):976 DISABLED toSslJdbcParam disable 2025-11-25 162848 INFO 2025-11-25 162848 info INFO main c.z.h.HikariDataSource(<init>):79 HikariPool-1 - Starting... 2025-11-25 162848 INFO 2025-11-25 162848 info INFO main c.z.h.HikariDataSource(<init>):81 HikariPool-1 - Start completed. 2025-11-25 162848 INFO 2025-11-25 162848 info INFO main c.z.h.HikariDataSource(close):349 HikariPool-1 - Shutdown initiated... 2025-11-25 162848 INFO 2025-11-25 162848 info INFO main c.z.h.HikariDataSource(close):351 HikariPool-1 - Shutdown completed. 2025-11-25 162848 INFO 2025-11-25 162848 info INFO main i.a.c.i.b.IntegrationRunner(runInternal):224 Completed integration: io.airbyte.cdk.integrations.base.ssh.SshWrappedSource 2025-11-25 162848 INFO 2025-11-25 162848 info INFO main i.a.i.s.p.PostgresSource(main):752 completed source: class io.airbyte.integrations.source.postgres.PostgresSource 2025-11-25 162848 INFO 2025-11-25 162848 info Checking for optional control message... 2025-11-25 162848 INFO 2025-11-25 162848 info Optional control message not found. Skipping... 2025-11-25 162849 INFO 2025-11-25 162849 info Writing output of decd338e-5647-4c0b-adf4-da0e75f5a750_436d097d-c41f-4775-bc3f-186803310e78_0_check to the doc store 2025-11-25 162849 INFO 2025-11-25 162849 info Marking workload decd338e-5647-4c0b-adf4-da0e75f5a750_436d097d-c41f-4775-bc3f-186803310e78_0_check as successful 2025-11-25 162849 INFO 2025-11-25 162849 info 2025-11-25 162849 INFO 2025-11-25 162849 info Deliberately exiting process with code 0. 2025-11-25 162849 INFO 2025-11-25 162849 info ----- END CHECK ----- 2025-11-25 162849 INFO 2025-11-25 162849 info what is the issue?

@kapa.ai what does "Days To Sync If History Is Full" mean for source-sftp-bulk? When do i know that the history is full?

@kapa.ai I have setup the airbyte on EKS cluster using the helm chart of version 8.5.0. When I am trying to create the custom connector after one time testing it doesn't allow me to update it again if I wanted to I need to create the new custom connector also it converts the UI into the yaml

🧵 help

@kapa.ai why would sync show success even though pod was never allocated? 2025-11-26 095322 platform INFO APPLY Stage: BUILD — (workloadId=d88dfecc-6847-4445-8733-c2d0ab51ff67_67_0_sync) 2025-11-26 095322 platform INFO APPLY Stage: CLAIM — (workloadId=d88dfecc-6847-4445-8733-c2d0ab51ff67_67_0_sync) 2025-11-26 095322 platform INFO Claimed: true for workload d88dfecc-6847-4445-8733-c2d0ab51ff67_67_0_sync via API in dataplane 2894b95b-f14c-446c-9c04-14eb422092fa (acf550f2-6155-4451-8db9-b62443826c92) 2025-11-26 095322 platform INFO APPLY Stage: LOAD_SHED — (workloadId=d88dfecc-6847-4445-8733-c2d0ab51ff67_67_0_sync) 2025-11-26 095322 platform INFO APPLY Stage: CHECK_STATUS — (workloadId=d88dfecc-6847-4445-8733-c2d0ab51ff67_67_0_sync) 2025-11-26 095322 platform INFO No pod found running for workload d88dfecc-6847-4445-8733-c2d0ab51ff67_67_0_sync 2025-11-26 095322 platform INFO APPLY Stage: MUTEX — (workloadId=d88dfecc-6847-4445-8733-c2d0ab51ff67_67_0_sync) 2025-11-26 095322 platform INFO Mutex key: d88dfecc-6847-4445-8733-c2d0ab51ff67 specified for workload: d88dfecc-6847-4445-8733-c2d0ab51ff67_67_0_sync. Attempting to delete existing pods... 2025-11-26 095322 platform INFO Existing pods for mutex key: d88dfecc-6847-4445-8733-c2d0ab51ff67 deleted. 2025-11-26 095322 platform INFO APPLY Stage: ARCHITECTURE — (workloadId=d88dfecc-6847-4445-8733-c2d0ab51ff67_67_0_sync) 2025-11-26 095322 platform INFO APPLY Stage: LAUNCH — (workloadId=d88dfecc-6847-4445-8733-c2d0ab51ff67_67_0_sync) 2025-11-26 095322 platform INFO [initContainer] image: airbyte/workload-init-container:2.0.1 resources: ResourceRequirements(claims=[], limits={memory=7530Mi, cpu=3.6723}, requests={memory=6348Mi, cpu=1.845}, additionalProperties={}) 2025-11-26 095322 platform INFO Launching replication pod: replication-job-67-attempt-0 (selectors = {kube/nodetype=airbyte-job}) with containers: 2025-11-26 095322 platform INFO [source] image: 454115844779.dkr.ecr.us-east-1.amazonaws.com/conn/teams-dev:e3395cc.4059 resources: ResourceRequirements(claims=[], limits={memory=7530Mi, cpu=3.6723}, requests={memory=6348Mi, cpu=1.845}, additionalProperties={}) 2025-11-26 095322 platform INFO [destination] image: 454115844779.dkr.ecr.us-east-1.amazonaws.com/conn/efsdest-dev:fe5e292.3541 resources: ResourceRequirements(claims=[], limits={memory=7530Mi, cpu=3.6723}, requests={memory=6348Mi, cpu=1.845}, additionalProperties={}) 2025-11-26 095322 platform INFO [orchestrator] image: airbyte/container-orchestrator:2.0.1 resources: ResourceRequirements(claims=[], limits={memory=7530Mi, cpu=3.6723}, requests={memory=6348Mi, cpu=1.845}, additionalProperties={}) 2025-11-26 102408 platform INFO ----- START POST REPLICATION OPERATIONS ----- 2025-11-26 102408 platform INFO No post-replication operation(s) to perform. 2025-11-26 102408 platform INFO ----- END POST REPLICATION OPERATIONS -----

This message was deleted.

@kapa.ai how do I increase memory requirements for the orchestrator container?

@kapa.ai I have an issue with replicating retool data in Snwoflake, The same tables don't have any issues in redshift. I tried to refresh the stream, clear the data, add a new connection with all streams, add a new connection to the only stream that is failing. where else should I look into?

@kapa.ai UpgradeFailed - Helm upgrade failed for release airbyte/airbyte with chart airbyte@1.8.5: template: airbyte/charts/workload-api-server/templates/secrets.yaml1460: executing "airbyte/charts/workload-api-server/templates/secrets.yaml" at <.Values.global.secrets>: wrong type for value; expected map[string]interface {}; got interface {}

does airbyte any group by at mongo db data source at CDC sync?

@kapa.ai where should one configure heart beat timeout? We have a long running sync, eventhough sync is running Airbyte has marked satus as success

@kapa.ai ive set up auth for service key authentication with my airbyte open source for sharepoint connector and i'm getting the following error:

facing one technical issue, We have setup the custom component for the reporting part, async stream is also working fine for the same. but the rest of the streams which are not using the custom connector, getting error while running them in builder UI. Could not load module

source_declarative_manifest.components

Hi @kapa.ai in one of my mongo db sync, i see this error but the pipeline is marked successful. Why would this be happening ? 2025-11-26 020051 platform ERROR Stage Pipeline Exception: io.airbyte.workload.launcher.pipeline.stages.model.StageError: io.airbyte.workers.exception.ResourceConstraintException: Unable to start the REPLICATION pod. This may be due to insufficient system resources. Please check available resources and try again. message: io.airbyte.workers.exception.ResourceConstraintException: Unable to start the REPLICATION pod. This may be due to insufficient system resources. Please check available resources and try again. stackTrace: [Ljava.lang.StackTraceElement;@5ea79bf4

I'm using the connector builder for a new connector. Its using Oauth2 authentication, where I need to request for a new access token using the client id + client secret + refresh token. I've properly configured it. However, once received a valid access token, I need to pass it in a custom header (x-amz-access-token) in the main request (not the standard bearer one). How to do this?

@kapa.ai after upgrading to 2.0.1, chart 1.9.1, I'm getting: 2025-11-26 164349,518 [io-executor-thread-2] ERROR i.g.o.k.KLogger$DefaultImpls(error):811 - Swallowed error. java.lang.NullPointerException: Cannot invoke "io.airbyte.config.SyncStats.getSourceStateMessagesEmitted()" because "totalStats" is null at io.airbyte.persistence.job.tracker.TrackingMetadata.generateJobAttemptMetadata(TrackingMetadata.kt:173) at io.airbyte.persistence.job.tracker.JobTracker.generateJobAttemptMetadata(JobTracker.kt:509) at io.airbyte.persistence.job.tracker.JobTracker.trackSync$lambda$7(JobTracker.kt:248) at io.airbyte.commons.lang.Exceptions.swallow(Exceptions.kt:67) at io.airbyte.persistence.job.tracker.JobTracker.trackSync(JobTracker.kt:217) at io.airbyte.commons.server.handlers.helpers.JobCreationAndStatusUpdateHelper.trackCompletion(JobCreationAndStatusUpdateHelper.kt:444) at io.airbyte.commons.server.handlers.JobsHandler.jobSuccessWithAttemptNumber(JobsHandler.kt:192) at io.airbyte.server.apis.controllers.JobsApiController.jobSuccessWithAttemptNumber$lambda$12(JobsApiController.kt:162) at io.airbyte.server.apis.ApiHelper.execute(ApiHelper.kt:32) at io.airbyte.server.apis.controllers.JobsApiController.jobSuccessWithAttemptNumber(JobsApiController.kt:161) at io.airbyte.server.apis.controllers.$JobsApiController$Definition$Exec.dispatch(Unknown Source) at io.micronaut.context.AbstractExecutableMethodsDefinition$DispatchedExecutableMethod.invokeUnsafe(AbstractExecutableMethodsDefinition.java:461) at io.micronaut.context.DefaultBeanContext$BeanContextUnsafeExecutionHandle.invokeUnsafe(DefaultBeanContext.java:4438) at io.micronaut.web.router.AbstractRouteMatch.execute(AbstractRouteMatch.java:272) at io.micronaut.web.router.DefaultUriRouteMatch.execute(DefaultUriRouteMatch.java:38) at io.micronaut.http.server.RouteExecutor.executeRouteAndConvertBody(RouteExecutor.java:465) at io.micronaut.http.server.RouteExecutor.lambda$callRoute$5(RouteExecutor.java:442) at io.micronaut.core.execution.ExecutionFlow.lambda$async$0(ExecutionFlow.java:92) at io.micronaut.core.propagation.PropagatedContext.lambda$wrap$3(PropagatedContext.java:232) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) at java.base/java.lang.Thread.run(Thread.java:1583)

@kapa.ai i installed airbyte 2.0.1 with helm charts v1 is it ok if i install it with helm charts v1 instead of with v2

@kapa.ai i want o access airbyte db and use those configurations for another airbyte setup in prod environment how can i do that??

@kapa.ai Error: couldn't find key WORKLOAD_API_BEARER_TOKEN in Secret airbyte/airbyte-airbyte-secrets. this is happening on airbyte 2.0.1 v1 helm chart

@kapa.ai is there any limit to the number of tables transfered from a database because when i am trying to make a sync upto 25 tables are getting synced more than that are not getting started why is that how to rectify this error

Configuration check failed State code: 08001; Message: The connection attempt failed.

@kapa.ai Configuration check failed State code: 08001; Message: The connection attempt failed.

does the airbyte marketplace rss connector support pagination?

I hit 403 'Forbidden' error when connecting to Airbyte from Airflow. What should I do?

@kapa.ai guide me through making an api through which I can hit build source, for restapis

@kapa.ai We have moved connectors to private docker repository and getting error "You must upgrade your platform version to use this connector version. Either downgrade your connector or upgrade platform to 0.63.7"