lors d une connexion hubspot bigquery j ai ces log...
# ask-ai
y
lors d une connexion hubspot bigquery j ai ces logs : 2025-07-22 082226,190 [pool-4-thread-4] ERROR i.a.w.i.VersionedAirbyteStreamFactory(internalLog$io_airbyte_airbyte_commons_worker):229 - ERROR main i.a.c.AirbyteConnectorRunnable(run):36 Failed class io.airbyte.cdk.load.write.WriteOperation operation execution. io.airbyte.cdk.TransientErrorException: Input was fully read, but some streams did not receive a terminal stream status message. If the destination did not encounter other errors, this likely indicates an error in the source or platform. Streams without a status message: [AirbytePRE.contacts, AirbytePRE.engagements_notes] at io.airbyte.cdk.load.state.SyncManager.markInputConsumed(SyncManager.kt:125) ~[io.airbyte.airbyte-cdk.bulk.core-bulk-cdk-core-load.jar:?] at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:704) ~[kotlinx-coroutines-core-jvm-1.10.2.jar:?] Stack Trace: io.airbyte.cdk.TransientErrorException: Input was fully read, but some streams did not receive a terminal stream status message. If the destination did not encounter other errors, this likely indicates an error in the source or platform. Streams without a status message: [AirbytePRE.contacts, AirbytePRE.engagements_notes] at io.airbyte.cdk.load.state.SyncManager.markInputConsumed(SyncManager.kt:125) at io.airbyte.cdk.load.state.PipelineEventBookkeepingRouter.close(PipelineEventBookkeepingRouter.kt:248) at io.airbyte.cdk.load.task.internal.InputConsumerTask.execute(InputConsumerTask.kt:130) at io.airbyte.cdk.load.task.internal.InputConsumerTask$execute$1.invokeSuspend(InputConsumerTask.kt) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:100) at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:124) at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:89) at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:586) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:820) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:717) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:704) 2025-07-22 082226,342 [pool-9-thread-1] ERROR i.a.c.o.w.i.LocalContainerAirbyteDestination(start$lambda$2):69 - io.airbyte.cdk.TransientErrorException: Input was fully read, but some streams did not receive a terminal stream status message. If the destination did not encounter other errors, this likely indicates an error in the source or platform. Streams without a status message: [AirbytePRE.contacts, AirbytePRE.engagements_notes] 2025-07-22 082226,342 [pool-9-thread-1] ERROR i.a.c.o.w.i.LocalContainerAirbyteDestination(start$lambda$2):69 - at io.airbyte.cdk.load.state.SyncManager.markInputConsumed(SyncManager.kt:125) 2025-07-22 082226,342 [pool-9-thread-1] ERROR i.a.c.o.w.i.LocalContainerAirbyteDestination(start$lambda$2):69 - at io.airbyte.cdk.load.state.PipelineEventBookkeepingRouter.close(PipelineEventBookkeepingRouter.kt:248) 2025-07-22 082226,342 [pool-9-thread-1] ERROR i.a.c.o.w.i.LocalContainerAirbyteDestination(start$lambda$2):69 - at io.airbyte.cdk.load.task.internal.InputConsumerTask.execute(InputConsumerTask.kt:130) 2025-07-22 082226,342 [pool-9-thread-1] ERROR i.a.c.o.w.i.LocalContainerAirbyteDestination(start$lambda$2):69 - at io.airbyte.cdk.load.task.internal.InputConsumerTask$execute$1.invokeSuspend(InputConsumerTask.kt) 2025-07-22 082226,342 [pool-9-thread-1] ERROR i.a.c.o.w.i.LocalContainerAirbyteDestination(start$lambda$2):69 - at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) 2025-07-22 082226,343 [pool-9-thread-1] ERROR i.a.c.o.w.i.LocalContainerAirbyteDestination(start$lambda$2):69 - at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:100) 2025-07-22 082226,343 [pool-9-thread-1] ERROR i.a.c.o.w.i.LocalContainerAirbyteDestination(start$lambda$2):69 - at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:124) 2025-07-22 082226,343 [pool-9-thread-1] ERROR i.a.c.o.w.i.LocalContainerAirbyteDestination(start$lambda$2):69 - at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:89) 2025-07-22 082226,343 [pool-9-thread-1] ERROR i.a.c.o.w.i.LocalContainerAirbyteDestination(start$lambda$2):69 - at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:586) 2025-07-22 082226,343 [pool-9-thread-1] ERROR i.a.c.o.w.i.LocalContainerAirbyteDestination(start$lambda$2):69 - at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:820) 2025-07-22 082226,343 [pool-9-thread-1] ERROR i.a.c.o.w.i.LocalContainerAirbyteDestination(start$lambda$2):69 - at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:717) 2025-07-22 082226,343 [pool-9-thread-1] ERROR i.a.c.o.w.i.LocalContainerAirbyteDestination(start$lambda$2):69 - at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:704) 2025-07-22 082227,670 [pool-4-thread-4] ERROR i.a.c.o.w.DestinationReader(run):59 - DestinationReader error: kotlinx.coroutines.JobCancellationException: Parent job is Cancelling Caused by: io.airbyte.workers.internal.exception.SourceException: Source process exited with non-zero exit code 1 at io.airbyte.container.orchestrator.worker.SourceReader.run(ReplicationTask.kt:196) at io.airbyte.container.orchestrator.worker.SourceReader$run$1.invokeSuspend(ReplicationTask.kt) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:100) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) at java.base/java.lang.Thread.run(Thread.java:1583) ERROR i.a.c.o.w.ReplicationWorker$run$2(invokeSuspend):120 - runJobs failed; recording failure but continuing to finish. io.airbyte.workers.internal.exception.SourceException: Source process exited with non-zero exit code 1 at io.airbyte.container.orchestrator.worker.SourceReader.run(ReplicationTask.kt:196) at io.airbyte.container.orchestrator.worker.SourceReader$run$1.invokeSuspend(ReplicationTask.kt) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:100) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) at java.base/java.lang.Thread.run(Thread.java:1583) ERROR i.a.c.o.w.ReplicationWorker(safeClose):77 - Error closing resource io.airbyte.container.orchestrator.worker.io.LocalContainerAirbyteDestination@6167a456; recording failure but continuing. io.airbyte.workers.exception.WorkerException: Destination process exit with code 1. This warning is normal if the job was cancelled. at io.airbyte.container.orchestrator.worker.io.LocalContainerAirbyteDestination.close(LocalContainerAirbyteDestination.kt:52) at io.airbyte.container.orchestrator.worker.ReplicationWorker.safeClose(ReplicationWorker.kt:75) at io.airbyte.container.orchestrator.worker.ReplicationWorker.run$io_airbyte_airbyte_container_orchestrator(ReplicationWorker.kt:137) at io.airbyte.container.orchestrator.worker.ReplicationWorker$run$1.invokeSuspend(ReplicationWorker.kt) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) at kotlinx.coroutines.internal.ScopeCoroutine.afterResume(Scopes.kt:35) at kotlinx.coroutines.AbstractCoroutine.resumeWith(AbstractCoroutine.kt:101) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:98) at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:263) at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:94) at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:70) at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source) at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:48) at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source) at io.airbyte.container.orchestrator.worker.ReplicationWorker.runReplicationBlocking(ReplicationWorker.kt:85) at io.airbyte.container.orchestrator.worker.ReplicationJobOrchestrator.run(ReplicationJobOrchestrator.kt:133) at io.airbyte.container.orchestrator.worker.ReplicationJobOrchestrator.runJob(ReplicationJobOrchestrator.kt:95) at io.airbyte.container.orchestrator.Application.run(Application.kt:61) at io.airbyte.container.orchestrator.ApplicationKt.main(Application.kt:31) 2025-07-22 082227,724 [main] ERROR i.a.c.o.w.ReplicationWorker(safeClose):77 - Error closing resource io.airbyte.container.orchestrator.worker.io.LocalContainerAirbyteSource@7ef7faa9; recording failure but continuing. io.airbyte.workers.exception.WorkerException: Source process exit with code 1. This warning is normal if the job was cancelled. at io.airbyte.container.orchestrator.worker.io.LocalContainerAirbyteSource.close(LocalContainerAirbyteSource.kt:39) at io.airbyte.container.orchestrator.worker.ReplicationWorker.safeClose(ReplicationWorker.kt:75) at io.airbyte.container.orchestrator.worker.ReplicationWorker.run$io_airbyte_airbyte_container_orchestrator(ReplicationWorker.kt:138) at io.airbyte.container.orchestrator.worker.ReplicationWorker$run$1.invokeSuspend(ReplicationWorker.kt) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) at kotlinx.coroutines.internal.ScopeCoroutine.afterResume(Scopes.kt:35) at kotlinx.coroutines.AbstractCoroutine.resumeWith(AbstractCoroutine.kt:101) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:98) at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:263) at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:94) at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:70) at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source) at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:48) at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source) at io.airbyte.container.orchestrator.worker.ReplicationWorker.runReplicationBlocking(ReplicationWorker.kt:85) at io.airbyte.container.orchestrator.worker.ReplicationJobOrchestrator.run(ReplicationJobOrchestrator.kt:133) at io.airbyte.container.orchestrator.worker.ReplicationJobOrchestrator.runJob(ReplicationJobOrchestrator.kt:95) at io.airbyte.container.orchestrator.Application.run(Application.kt:61) at io.airbyte.container.orchestrator.ApplicationKt.main(Application.kt:31) le mode de sync est full refresh overwrite comment je peux resoudre ce probleme ?
k
Le message d’erreur que vous rencontrez lors de la synchronisation HubSpot → BigQuery indique que certaines streams (ici,
AirbytePRE.contacts
et
AirbytePRE.engagements_notes
) n’ont pas reçu de message de statut terminal, ce qui suggère un problème côté source ou plateforme, surtout si la destination ne signale pas d’autres erreurs. Ce type de problème est connu dans les synchronisations HubSpot → BigQuery, notamment lorsque le mode de synchronisation est “Full Refresh | Overwrite” et que la source (HubSpot) contient beaucoup de propriétés personnalisées. Cela peut entraîner des requêtes SQL trop longues ou des problèmes de gestion de flux côté Airbyte, comme décrit dans l’issue #36303. Voici les recommandations issues des connaissances Airbyte : 1. Réduire le nombre de propriétés synchronisées Désélectionnez les propriétés non essentielles dans la configuration du stream HubSpot, en particulier pour les objets volumineux comme Contacts. Cela permet de contourner les limitations liées à la taille des requêtes générées et à la gestion des flux. C’est une opération manuelle, mais elle a permis de résoudre ce problème pour d’autres utilisateurs. 2. Changer de mode de synchronisation Si possible, passez du mode “Full Refresh | Overwrite” à “Incremental + Dedup”. Ce mode est moins susceptible de générer des requêtes SQL trop volumineuses et peut améliorer la stabilité de la synchronisation. 3. Mettre à jour le connecteur BigQuery Essayez de migrer vers la version 3.0.1+ du connecteur BigQuery, qui utilise un nouveau paradigme de chargement direct (“Direct Load”) et ne crée plus les tables “raw” dans
airbyte_internal
. Cela permet d’éviter la génération de requêtes SQL très longues et améliore la compatibilité avec des schémas complexes. Attention, cette migration est une breaking change : suivez attentivement le [BigQuery Migration Guide](https://docs.airbyte.com/integrations/destinations/bigquery-migrations) avant de procéder à la mise à jour. En résumé, commencez par réduire le nombre de propriétés synchronisées, puis testez un mode de synchronisation incrémental, et enfin envisagez la migration vers la nouvelle version du connecteur BigQuery si le problème persiste. Ces solutions sont recommandées par l’équipe Airbyte pour ce type de blocage : [HubSpot to BigQuery destination blocked by known issue (36303)](https://github.com/airbytehq/airbyte/issues/62090). Si le problème persiste malgré ces actions, il est probable qu’il s’agisse d’une limitation structurelle de la plateforme ou d’un bug non résolu côté Airbyte.