Hi Folks, I have self hosted airbyte. I am trying to delete one of my connection, but it is not deleted. In it logs, it showing that

NEED_STATE_VALLIDATATION= true

env variable error. How can i delete this connection? Can you guys please help me on this

Hi, we have a custom source connector using a custom Postgres destination which has been running successfully up until we upgraded from Airbyte version 0.38.2 to 0.43.2 last week. We now get an error (as below) on syncing the data, which we have no idea what it means.

Additional Failure Information: message='activity timeout', timeoutType=TIMEOUT_TYPE_HEARTBEAT

We have never observed this failure message before the upgrade, and the connector worked without issue. Can you please shed some light on this? Full logs are as below.

2023-05-01 06:15:35 [32mINFO[m i.a.w.g.DefaultReplicationWorker(lambda$readFromSrcAndWriteToDstRunnable$7):423 - Records read: 80000 (19 MB)
2023-05-01 06:15:50 [32mINFO[m i.a.w.t.TemporalAttemptExecution(lambda$getCancellationChecker$6):231 - Running sync worker cancellation...
2023-05-01 06:15:50 [32mINFO[m i.a.w.g.DefaultReplicationWorker(cancel):722 - Cancelling replication worker...
2023-05-01 06:15:52 [32mINFO[m i.a.w.g.DefaultReplicationWorker(lambda$readFromSrcAndWriteToDstRunnable$7):423 - Records read: 85000 (20 MB)
2023-05-01 06:16:00 [32mINFO[m i.a.w.g.DefaultReplicationWorker(cancel):731 - Cancelling destination...
2023-05-01 06:16:00 [32mINFO[m i.a.w.i.DefaultAirbyteDestination(cancel):152 - Attempting to cancel destination process...
2023-05-01 06:16:00 [32mINFO[m i.a.w.i.DefaultAirbyteDestination(cancel):157 - Destination process exists, cancelling...
2023-05-01 06:16:00 [32mINFO[m i.a.w.p.KubePodProcess(destroy):692 - (pod: airbyte / xxxxxxxx-write-569069-0-tvtyw) - Destroying Kube process.
2023-05-01 06:16:00 [32mINFO[m i.a.w.g.DefaultReplicationWorker(lambda$readFromSrcAndWriteToDstRunnable$7):436 - Total records read: 86771 (21 MB)
2023-05-01 06:16:00 [33mWARN[m i.a.w.g.DefaultReplicationWorker(lambda$readFromSrcAndWriteToDstRunnable$6):446 - Source d2a0d30a-5831-4661-b92e-b2f1ea9a6577 has unexpected fields [Otherweblinks] in stream _gender_statistics_series_metadata
2023-05-01 06:16:00 [32mINFO[m i.a.w.i.HeartbeatTimeoutChaperone(runWithHeartbeatThread):111 - thread status... heartbeat thread: false , replication thread: true
2023-05-01 06:16:00 [32mINFO[m i.a.w.g.DefaultReplicationWorker(replicate):262 - Waiting for source and destination threads to complete.
2023-05-01 06:16:00 [32mINFO[m i.a.w.g.DefaultReplicationWorker(replicate):267 - One of source or destination thread complete. Waiting on the other.
2023-05-01 06:16:00 [32mINFO[m i.a.w.g.DefaultReplicationWorker(replicate):269 - Source and destination threads complete.
2023-05-01 06:16:00 [32mINFO[m i.a.w.p.KubePodProcess(close):760 - (pod: airbyte / xxxxxxxx-write-569069-0-tvtyw) - Closed all resources for pod
2023-05-01 06:16:00 [32mINFO[m i.a.w.p.KubePodProcess(destroy):698 - (pod: airbyte / xxxxxxxx-write-569069-0-tvtyw) - Destroyed Kube process.
2023-05-01 06:16:00 [32mINFO[m i.a.w.i.DefaultAirbyteDestination(cancel):159 - Cancelled destination process!
2023-05-01 06:16:00 [32mINFO[m i.a.w.g.DefaultReplicationWorker(cancel):739 - Cancelling source...
2023-05-01 06:16:00 [33mWARN[m i.a.c.i.LineGobbler(voidCall):154 - airbyte-destination gobbler IOException: Socket closed. Typically happens when cancelling a job.
2023-05-01 06:16:00 [32mINFO[m i.a.w.i.DefaultAirbyteSource(cancel):157 - Attempting to cancel source process...
2023-05-01 06:16:00 [32mINFO[m i.a.w.i.DefaultAirbyteSource(cancel):162 - Source process exists, cancelling...
2023-05-01 06:16:00 [32mINFO[m i.a.w.p.KubePodProcess(destroy):692 - (pod: airbyte / xxxxxxxx-read-569069-0-oztux) - Destroying Kube process.
2023-05-01 06:16:00 [32mINFO[m i.a.w.p.KubePodProcess(close):760 - (pod: airbyte / xxxxxxxx-read-569069-0-oztux) - Closed all resources for pod
2023-05-01 06:16:00 [33mWARN[m i.a.c.i.LineGobbler(voidCall):154 - airbyte-source gobbler IOException: Socket closed. Typically happens when cancelling a job.
2023-05-01 06:16:00 [32mINFO[m i.a.w.p.KubePodProcess(destroy):698 - (pod: airbyte / xxxxxxxx-read-569069-0-oztux) - Destroyed Kube process.
2023-05-01 06:16:00 [32mINFO[m i.a.w.i.DefaultAirbyteSource(cancel):164 - Cancelled source process!
2023-05-01 06:16:00 [32mINFO[m i.a.w.t.TemporalAttemptExecution(lambda$getCancellationChecker$6):235 - Interrupting worker thread...
2023-05-01 06:16:00 [32mINFO[m i.a.w.t.TemporalAttemptExecution(lambda$getCancellationChecker$6):238 - Cancelling completable future...
2023-05-01 06:16:00 [33mWARN[m i.a.c.t.CancellationHandler$TemporalCancellationHandler(checkAndHandleCancellation):60 - Job either timed out or was cancelled.
2023-05-01 06:16:00 [33mWARN[m i.a.c.t.CancellationHandler$TemporalCancellationHandler(checkAndHandleCancellation):60 - Job either timed out or was cancelled.
2023-05-01 06:16:00 [1;31mERROR[m i.a.w.WorkerUtils(gentleClose):72 - Exception while while waiting for process to finish
java.lang.InterruptedException: sleep interrupted
	at java.lang.Thread.sleep0(Native Method) ~[?:?]
	at java.lang.Thread.sleep(Thread.java:465) ~[?:?]
	at java.lang.Process.waitFor(Process.java:468) ~[?:?]
	at io.airbyte.workers.WorkerUtils.gentleClose(WorkerUtils.java:69) ~[io.airbyte-airbyte-commons-worker-0.43.2.jar:?]
	at io.airbyte.workers.internal.DefaultAirbyteSource.close(DefaultAirbyteSource.java:143) ~[io.airbyte-airbyte-commons-worker-0.43.2.jar:?]
	at io.airbyte.workers.general.DefaultReplicationWorker.replicate(DefaultReplicationWorker.java:271) ~[io.airbyte-airbyte-commons-worker-0.43.2.jar:?]
	at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:187) ~[io.airbyte-airbyte-commons-worker-0.43.2.jar:?]
	at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:91) ~[io.airbyte-airbyte-commons-worker-0.43.2.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$5(TemporalAttemptExecution.java:195) ~[io.airbyte-airbyte-workers-0.43.2.jar:?]
	at java.lang.Thread.run(Thread.java:1589) ~[?:?]
2023-05-01 06:16:00 [32mINFO[m i.a.w.t.TemporalAttemptExecution(get):163 - Stopping cancellation check scheduling...
2023-05-01 06:16:00 [32mINFO[m i.a.c.t.TemporalUtils(withBackgroundHeartbeat):307 - Stopping temporal heartbeating...
2023-05-01 06:16:00 [32mINFO[m i.a.a.c.AirbyteApiClient(retryWithJitter):179 - Attempt 0 to Flush States from SyncPersistenceImpl
2023-05-01 06:16:00 [33mWARN[m i.t.i.a.ActivityTaskExecutors$BaseActivityTaskExecutor(execute):114 - Activity failure. ActivityId=16064d4c-7b6f-369a-8e90-c919ac9694c8, activityType=Replicate, attempt=1
java.lang.RuntimeException: java.util.concurrent.CancellationException
	at io.airbyte.commons.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:305) ~[io.airbyte-airbyte-commons-temporal-0.43.2.jar:?]
	at io.airbyte.workers.temporal.sync.ReplicationActivityImpl.replicate(ReplicationActivityImpl.java:157) ~[io.airbyte-airbyte-workers-0.43.2.jar:?]
	at jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) ~[?:?]
	at java.lang.reflect.Method.invoke(Method.java:578) ~[?:?]
	at io.temporal.internal.activity.RootActivityInboundCallsInterceptor$POJOActivityInboundCallsInterceptor.executeActivity(RootActivityInboundCallsInterceptor.java:64) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.activity.RootActivityInboundCallsInterceptor.execute(RootActivityInboundCallsInterceptor.java:43) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.activity.ActivityTaskExecutors$BaseActivityTaskExecutor.execute(ActivityTaskExecutors.java:95) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.activity.ActivityTaskHandlerImpl.handle(ActivityTaskHandlerImpl.java:92) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handleActivity(ActivityWorker.java:241) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:206) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:179) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:93) ~[temporal-sdk-1.17.0.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
	at java.lang.Thread.run(Thread.java:1589) ~[?:?]
Caused by: java.util.concurrent.CancellationException
	at java.util.concurrent.CompletableFuture.cancel(CompletableFuture.java:2510) ~[?:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getCancellationChecker$6(TemporalAttemptExecution.java:241) ~[io.airbyte-airbyte-workers-0.43.2.jar:?]
	at io.airbyte.commons.temporal.CancellationHandler$TemporalCancellationHandler.checkAndHandleCancellation(CancellationHandler.java:59) ~[io.airbyte-airbyte-commons-temporal-0.43.2.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getCancellationChecker$7(TemporalAttemptExecution.java:244) ~[io.airbyte-airbyte-workers-0.43.2.jar:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:577) ~[?:?]
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:358) ~[?:?]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) ~[?:?]
	... 3 more
2023-05-01 06:16:00 [33mWARN[m i.t.i.w.ActivityWorker$TaskHandlerImpl(logExceptionDuringResultReporting):365 - Failure during reporting of activity result to the server. ActivityId = 16064d4c-7b6f-369a-8e90-c919ac9694c8, ActivityType = Replicate, WorkflowId=sync_569069, WorkflowType=SyncWorkflow, RunId=4567058d-17ca-4c3a-ae95-6cd03eaf7297
io.grpc.StatusRuntimeException: NOT_FOUND: workflow execution already completed
	at io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:271) ~[grpc-stub-1.52.1.jar:1.52.1]
	at io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:252) ~[grpc-stub-1.52.1.jar:1.52.1]
	at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:165) ~[grpc-stub-1.52.1.jar:1.52.1]
	at io.temporal.api.workflowservice.v1.WorkflowServiceGrpc$WorkflowServiceBlockingStub.respondActivityTaskFailed(WorkflowServiceGrpc.java:3866) ~[temporal-serviceclient-1.17.0.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.lambda$sendReply$1(ActivityWorker.java:320) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.retryer.GrpcRetryer.lambda$retry$0(GrpcRetryer.java:52) ~[temporal-serviceclient-1.17.0.jar:?]
	at io.temporal.internal.retryer.GrpcSyncRetryer.retry(GrpcSyncRetryer.java:67) ~[temporal-serviceclient-1.17.0.jar:?]
	at io.temporal.internal.retryer.GrpcRetryer.retryWithResult(GrpcRetryer.java:60) ~[temporal-serviceclient-1.17.0.jar:?]
	at io.temporal.internal.retryer.GrpcRetryer.retry(GrpcRetryer.java:50) ~[temporal-serviceclient-1.17.0.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.sendReply(ActivityWorker.java:315) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handleActivity(ActivityWorker.java:252) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:206) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:179) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:93) ~[temporal-sdk-1.17.0.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
	at java.lang.Thread.run(Thread.java:1589) ~[?:?]

Hi, I have been using airbyte to sync data from google sheets to firestore. The instance is deployed on EC2 and I'm sshing it to localhost. Everything was working fine until yesterday, but today the screen does not load and this errors were on console.

Hi, I have using airbyte sycn mysql to aws s3 for a while.Now I have lots of zip files in sftp server, Could i use airbyte sync zip files in sftp to aws s3.Thanks

And I need to download lots of zip files from web by https ,then upload to aws s3. How can i use airbyte to finish the process?

Hi there! Anybody here have experiencing syncing large Postgres tables via the Postgres source? I am attempting to sync a 500m+ row Postgres table to an empty BigQuery destination, but the job seems to cease responding at around the 420m row mark, wondering if anybody has dealt with similar problems

Hi there Airbyte community, I am having issues creating connections on Airbyte deployed on Kubernetes (EKS) I have tried to make connections to both Snowflake and Postgres but I keep getting this error which I haven't seen in searching through this slack. I am having the following error:

Configuration check failed
Failed to run connection tests.

The logs for the connection look something like this (only 3 lines each time):

2023-05-01 22:02:10 [32mINFO[m i.a.w.t.TemporalAttemptExecution(get):138 - Cloud storage job log path: /workspace/08959636-4a24-4339-9498-ab44ee0fadd1/0/logs.log
2023-05-01 22:02:10 [32mINFO[m i.a.w.t.TemporalAttemptExecution(get):141 - Executing worker wrapper. Airbyte version: 0.44.2
2023-05-01 22:02:10 [32mINFO[m i.a.a.c.AirbyteApiClient(retryWithJitter):179 - Attempt 0 to save workflow id for cancellation

This happens with both sources, it seems like it isn't even getting to the step to create the connection. Has anyone encountered a similar issue before? I think it might be something to do with the underlying deployment not working but haven't seen this happen before in Airbyte with Docker.

Hey Airbyte friends, I'm having an issue with the alpha Netsuite connector that seems very similar to the issue experienced in this thread but I wasn't clear on if there was any resolution. Essentially when attempted to set up Netsuite as a source, the credentials I use pass all connection tests. Then, when attempting to add a destination, I keep getting an error message that reads

Failed to fetch schema. Please try again 
Error: non-json response

Does anyone have any advice on how to debug?

Hello team, I am new to dbt, I need to transfer data from mysql to bigquery after that i need to do typecasting in the bigquery using dbt How to dbt here, can anyone explain me in detail

Hello, I'm using airbyte version 0.44.4 in a source sql server connection and destination postgresql, using cdc, it's returning the following error: Stack Trace: java.lang.RuntimeException: java.nio.file.NoSuchFileException: /tmp/cdc-db-history6084744130435399984/dbhistory.dat at io.airbyte.integrations.debezium.internals.AirbyteSchemaHistoryStorage.read(AirbyteSchemaHistoryStorage.java:57) at io.airbyte.integrations.debezium.internals.DebeziumStateDecoratingIterator.createStateMessage(DebeziumStateDecoratingIterator.java:201) at io.airbyte.integrations.debezium.internals.DebeziumStateDecoratingIterator.computeNext(DebeziumStateDecoratingIterator.java:174) at io.airbyte.integrations.debezium.internals.DebeziumStateDecoratingIterator.computeNext(DebeziumStateDecoratingIterator.java:28) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:146) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:141) As I understand it, I ran the first sync as Incremental Sync - Deduped History, and then changed it to incremental - append, it looks like it's trying to find /tmp/cdc-db-history.... Any tips on how to resolve?

Hi, I have deployed Airbyte on AWS, and I'am trying to connect to the PostgreSQL in AWS, but encountering this error 'Message: HikariPool-1 - Connection is not available, request timed out after 10002ms', and I attached the log, can anyone help out? Thanks

i connected amazon ads for getting amazon attribution data..but none of the metrics/dimensions like campaign, clicks, dvp are present in the tables?

This message was deleted.

There is solution to connect to Amazon DocumentDB (MongoDB like) using MongoDB connector? I am not able to connect to DocumentDB because the certificate. I found some posts and PR's about this, but I did'nt see solution for this case.

Anyone successfully deployed on GCE VM with a postgres external db? Just stuck in a loop of the bootloader and temporal failing

Hey octavia wave! We will be upgrading our

destination-snowflake

connector and will be removing the AzureBlobStorage as a loading option. If you are using AzureBlobStorage as your

destination-snowflake

loading option do NOT upgrade your destination-snowflake version above 0.4.63 until updating your loading method. We recommend using Internal Staging. Thanks team!

Hi, i’m trying to sync postgres to #databricks • i’cant find Source: Incremental Dest: Deduped + history Mode. and i think it will cost more if we go with full refreash, anyone have idea about move data to databricks with airbyte?

Hello, I am using Airbyte on EC2 with Docker. I am currently trying to connect MySQL CDC to Snowflake, but I am getting an error. The error message is as follows. I found out that I can fix the problem by modifying the 'blobSizeLimitError' and 'blobSizeLimitWarn' in Temporal. Can you please let me know how to modify these settings in my Airbyte environment? Thank you. airbyte version : 0.44.3 mysql : 5.7 mysql source : 2.0.19 snowflake destination : 0.4.63

2023-05-02 22:22:24 ERROR i.a.w.t.s.ConnectionManagerWorkflowImpl(runMandatoryActivityWithOutput):595 - [ACTIVITY-FAILURE] Connection c77f545d-4206-465b-936a-c24460f6df92 failed to run an activity.(SyncInputWithAttemptNumber).  Connection manager workflow will be restarted after a delay of PT10M.
io.temporal.failure.ActivityFailure: scheduledEventId=66, startedEventId=67, activityType='GetSyncWorkflowInputWithAttemptNumber', activityId='8d1285c5-820c-3851-8d7a-63a5bd43c3f4', identity='1@0d7d251db3d6', retryState=RETRY_STATE_NON_RETRYABLE_FAILURE
	at java.lang.Thread.getStackTrace(Thread.java:2550) ~[?:?]
	at io.temporal.internal.sync.ActivityStubBase.execute(ActivityStubBase.java:49) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.sync.ActivityInvocationHandler.lambda$getActivityFunc$0(ActivityInvocationHandler.java:78) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.sync.ActivityInvocationHandlerBase.invoke(ActivityInvocationHandlerBase.java:60) ~[temporal-sdk-1.17.0.jar:?]
	at jdk.proxy2.$Proxy82.getSyncWorkflowInputWithAttemptNumber(Unknown Source) ~[?:?]
	at io.airbyte.workers.temporal.scheduling.ConnectionManagerWorkflowImpl.runMandatoryActivityWithOutput(ConnectionManagerWorkflowImpl.java:593) ~[io.airbyte-airbyte-workers-0.44.3.jar:?]
	at io.airbyte.workers.temporal.scheduling.ConnectionManagerWorkflowImpl.getJobInput(ConnectionManagerWorkflowImpl.java:748) ~[io.airbyte-airbyte-workers-0.44.3.jar:?]
	at io.airbyte.workers.temporal.scheduling.ConnectionManagerWorkflowImpl.lambda$generateSyncWorkflowRunnable$1(ConnectionManagerWorkflowImpl.java:247) ~[io.airbyte-airbyte-workers-0.44.3.jar:?]
	at io.temporal.internal.sync.CancellationScopeImpl.run(CancellationScopeImpl.java:102) ~[temporal-sdk-1.17.0.jar:?]
	at io.airbyte.workers.temporal.scheduling.ConnectionManagerWorkflowImpl.run(ConnectionManagerWorkflowImpl.java:151) ~[io.airbyte-airbyte-workers-0.44.3.jar:?]
	at ConnectionManagerWorkflowImplProxy.run$accessor$wGtze6iR(Unknown Source) ~[?:?]
	at ConnectionManagerWorkflowImplProxy$auxiliary$X9sWbE9c.call(Unknown Source) ~[?:?]
	at io.airbyte.workers.temporal.support.TemporalActivityStubInterceptor.execute(TemporalActivityStubInterceptor.java:79) ~[io.airbyte-airbyte-workers-0.44.3.jar:?]
	at ConnectionManagerWorkflowImplProxy.run(Unknown Source) ~[?:?]
	at jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) ~[?:?]
	at java.lang.reflect.Method.invoke(Method.java:578) ~[?:?]
	at io.temporal.internal.sync.POJOWorkflowImplementationFactory$POJOWorkflowImplementation$RootWorkflowInboundCallsInterceptor.execute(POJOWorkflowImplementationFactory.java:302) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.sync.POJOWorkflowImplementationFactory$POJOWorkflowImplementation.execute(POJOWorkflowImplementationFactory.java:277) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.sync.WorkflowExecuteRunnable.run(WorkflowExecuteRunnable.java:71) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.sync.SyncWorkflow.lambda$start$0(SyncWorkflow.java:116) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.sync.CancellationScopeImpl.run(CancellationScopeImpl.java:102) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.sync.WorkflowThreadImpl$RunnableWrapper.run(WorkflowThreadImpl.java:106) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.worker.ActiveThreadReportingExecutor.lambda$submit$0(ActiveThreadReportingExecutor.java:53) ~[temporal-sdk-1.17.0.jar:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:577) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
	at java.lang.Thread.run(Thread.java:1589) ~[?:?]
Caused by: io.temporal.failure.ServerFailure: Complete result exceeds size limit.
	at io.temporal.failure.FailureConverter.failureToExceptionImpl(FailureConverter.java:136) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.failure.FailureConverter.failureToException(FailureConverter.java:79) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.failure.FailureConverter.failureToExceptionImpl(FailureConverter.java:93) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.failure.FailureConverter.failureToException(FailureConverter.java:79) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.sync.SyncWorkflowContext$ActivityCallback.lambda$invoke$0(SyncWorkflowContext.java:292) ~[temporal-sdk-1.17.0.jar:?]
	... 8 more

Airbyte version: 0.43 Source: Postgres v2.0.28 Destination: BigQuery v1.2.20 Using GCS for temporary data? yes I am getting this sync error intermittently (usually syncs succeeded, but sometimes they fail, all 3 attempts)

com.amazonaws.SdkClientException: Unable to execute HTTP request: Broken pipe

Attaching full log here. The above error is on line 60729.

Has anyone succeeded in pulling tasks from Asana? I know it's still in alpha, but it seems the pull misses half the tasks. Why could that be?

Hello! I am syncing data from postgres to bigquery using incremental + deduped mode. Sometimes sync is emitting too much records and writing some data from scratch, it happens 1 time out of 10 syncs and it is pretty random. What could be the reason for this?

Hi Team, We are seeing web-hook trigger from Airbyte twice for same connection success. below is logs after download succeeds. Airbyte version is 0.42.0 has anyone observed this ?

2023-04-19 12:05:43 INFO i.a.w.t.s.ReplicationActivityImpl(lambda$replicate$4):264 - Sync summary length: 2897
2023-04-19 12:05:43 INFO i.a.c.t.TemporalUtils(withBackgroundHeartbeat):307 - Stopping temporal heartbeating...
2023-04-19 12:06:22 INFO i.a.n.SlackNotificationClient(notify):151 - Successful notification (200): {"data":{"text":"Your connection from File (CSV, JSON, Excel, Feather, Parquet) to Custom Destination succeeded\nThis was for sync started on Wednesday, April 19, 2023 at 12:05:29 PM Coordinated Universal Time, running for 14 seconds.\n\nYou can access its logs here: <http://localhost:8000/workspaces/21ecbb67-50ea-45d5-bc07-5acbc0069559/connections/14d59d7f-5cb5-4baf-97a7-818bd4255ebb>\n\nJob ID: 349"},"status":{"code":200,"result":"success"}}
2023-04-19 12:06:22 WARN i.t.i.w.ActivityWorker$TaskHandlerImpl(logExceptionDuringResultReporting):365 - Failure during reporting of activity result to the server. ActivityId = 2723ff6e-8860-36ae-9915-8dd37b906129, ActivityType = JobSuccessWithAttemptNumber, WorkflowId=connection_manager_14d59d7f-5cb5-4baf-97a7-818bd4255ebb, WorkflowType=ConnectionManagerWorkflow, RunId=b30d1854-239f-46dd-9956-5af06089642d
io.grpc.StatusRuntimeException: NOT_FOUND: invalid activityID or activity already timed out or invoking workflow is completed
	at io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:271) ~[grpc-stub-1.52.1.jar:1.52.1]
	at io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:252) ~[grpc-stub-1.52.1.jar:1.52.1]
	at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:165) ~[grpc-stub-1.52.1.jar:1.52.1]
	at io.temporal.api.workflowservice.v1.WorkflowServiceGrpc$WorkflowServiceBlockingStub.respondActivityTaskCompleted(WorkflowServiceGrpc.java:3840) ~[temporal-serviceclient-1.17.0.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.lambda$sendReply$0(ActivityWorker.java:303) ~[temporal-sdk-1.17.0.jar:?]
2023-04-19 12:06:49 ERROR i.a.p.j.JobNotifier(notifyJob):133 - Failed to notify: io.airbyte.config.Notification@209426fd[notificationType=slack,sendOnSuccess=true,sendOnFailure=true,slackConfiguration=io.airbyte.config.SlackNotificationConfiguration@646941c3[webhook=<https://dev.weav.ai/ingestion-service/api/v1/connectors/sync_webhook],customerioConfiguration=><null>,additionalProperties={}] due to an exception
java.io.IOException: Failed to deliver notification (500): {"correlation_id":"2106942d-e416-4677-80fb-aa624e10c185","data":"fail","status":{"code":500,"result":"fail"}}

Hi All. Do we have an open source connector for logistics company such as DHL or FedEx? Asking this because I want to download the delivery data and upload them to our data warehouse.

Am I correct that the GA4 connector still does not support filters?

Hello everyone, Attempting to create Airbyte connector using Java wherein whilst executing the standard test cases in our local environment some of which are failing. It is looking for files "destination_config.json" and "destination_catalog.json". These files does not exist in airbyte repo. When checked for logs, found that it is searching for these files under directory "/data/job". This directory generated during test case execution using below command : {"type":"LOG","log":{"level":"INFO","message":"INFO i.a.w.p.DockerProcessFactory(create):176 Preparing command: docker run --rm --init -i -w /data/job --log-driver none --name destination-vertica-check-0-0-zjnzg --network host -v /tmp/airbyte_tests/test14836968802016406834:/data -v /tmp/airbyte_tests/output4991573947145016338:/local -e USE_STREAM_CAPABLE_STATE=false -e FIELD_SELECTION_WORKSPACES= -e APPLY_FIELD_SELECTION=false -e WORKER_CONNECTOR_IMAGE=airbyte/destination-vertica:dev -e WORKER_JOB_ATTEMPT=0 -e AUTO_DETECT_SCHEMA=true -e WORKER_JOB_ID=0 airbyte/destination-vertica:dev check --config source_config.json"}} Also, our PR needs review. Kindly suggest if that can be reviewed by someone from Airbyte team : https://github.com/airbytehq/airbyte/pull/25682 Many Thanks in advance. Kind Regards/Vivek

Hi team, is there still a way to use the HTTPS request connector? We're just trying to setup a simple API pull from a single endpoint to BambooHR but i think the AIrbyte CDK is a bit too.. technical for us

Hi Team, Facing isuue with salesforce as source and elastic as destination with sync mode incremental-sync append getting added +1 without doing any change or update or add ? is this a bug from airbyte ?? Please let me know....

hey guys, having trouble creating a connection between source and destination. below are the details: source: redshift destination: s3 error:

Internal message: java.lang.NullPointerException: null value in entry: isNullable=null
Failure type: system_error

Stacktrace
java.lang.NullPointerException: null value in entry: isNullable=null
	at com.google.common.collect.CollectPreconditions.checkEntryNotNull(CollectPreconditions.java:33)
	at com.google.common.collect.ImmutableMapEntry.<init>(ImmutableMapEntry.java:54)
	at com.google.common.collect.ImmutableMap.entryOf(ImmutableMap.java:339)
	at com.google.common.collect.ImmutableMap$Builder.put(ImmutableMap.java:449)
	at io.airbyte.integrations.source.jdbc.AbstractJdbcSource.getColumnMetadata(AbstractJdbcSource.java:268)
	at io.airbyte.db.jdbc.JdbcDatabase$1.tryAdvance(JdbcDatabase.java:81)
	at java.base/java.util.Spliterator.forEachRemaining(Spliterator.java:332)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
	at io.airbyte.db.jdbc.DefaultJdbcDatabase.bufferedResultSetQuery(DefaultJdbcDatabase.java:56)
	at io.airbyte.integrations.source.jdbc.AbstractJdbcSource.discoverInternal(AbstractJdbcSource.java:191)
	at io.airbyte.integrations.source.jdbc.AbstractJdbcSource.discoverInternal(AbstractJdbcSource.java:275)
	at io.airbyte.integrations.source.redshift.RedshiftSource.discoverInternal(RedshiftSource.java:98)
	at io.airbyte.integrations.source.redshift.RedshiftSource.discoverInternal(RedshiftSource.java:30)
	at io.airbyte.integrations.source.relationaldb.AbstractDbSource.discoverWithoutSystemTables(AbstractDbSource.java:255)
	at io.airbyte.integrations.source.relationaldb.AbstractDbSource.getTables(AbstractDbSource.java:509)
	at io.airbyte.integrations.source.relationaldb.AbstractDbSource.discover(AbstractDbSource.java:110)

how can we fix this?

hey guys, having trouble creating a connection between source and destination. below are the details: source: redshift destination: s3 error:

Internal message: java.lang.NullPointerException: null value in entry: isNullable=null
Failure type: system_error

Stacktrace
java.lang.NullPointerException: null value in entry: isNullable=null

let me know if you require full stack trace

hi guys, source: mysql destination: postgres Error: io.airbyte.workers.exception.WorkerException: Could not find image: airbyte/normalization-postgres:0.2.24 I can't even find the image in dockerhub!