https://linen.dev logo
Join Slack
Powered by
Hi everyone! I have a problem with the Google Ads ...
# replication-troubleshooting
y
Hi everyone! I have a problem with the Google Ads connector in the OSS version of Airbyte (2.0.1). I am using it to load campaign data to Redshift. Up till December 11th it worked fine, then at one point it just stopped loading data. What I see is that the sync starts to load data, loads a few thousand rows, then just stays in the running state without loading anything and i cannot see in the log the problem. I tried to do a full reload of the campaigns table too, same problem. It loads ~1360000 rows, then just keeps logging the same fragment and not loading any more records. Airbyte version 2.0.1, running on K8s deployed via the Helm Chart. Source connector: Google Ads v.4.1.3 Destination: Redshift v3.5.3 I will post the section of the logs in the comments. I've tried reauthenticating the source connector, that didn't help
Copy code
[...]
2025-12-15 12:33:33 replication-orchestrator INFO Records read: 1360000 (2 GB)
2025-12-15 12:34:28 destination INFO pool-3-thread-1 i.a.c.i.d.a.b.BufferManager(printQueueInfo):94 [ASYNC QUEUE INFO] Global: max: 371.22 MB, allocated: 20.07 MB (20.068470001220703 MB), %% used: 0.054061035443982025 | Queue `ab_3_campaign`, num records: 4625, num bytes: 9.97 MB, allocated bytes: 10.07 MB | State Manager memory usage: Allocated: 9 MB, Used: -1949 bytes, percentage Used -1.859056787650979E-4
2025-12-15 12:34:28 destination INFO pool-6-thread-1 i.a.c.i.d.a.FlushWorkers(printWorkerInfo):127 [ASYNC WORKER INFO] Pool queue size: 0, Active threads: 0
2025-12-15 12:35:28 destination INFO pool-3-thread-1 i.a.c.i.d.a.b.BufferManager(printQueueInfo):94 [ASYNC QUEUE INFO] Global: max: 371.22 MB, allocated: 20.07 MB (20.068470001220703 MB), %% used: 0.054061035443982025 | Queue `ab_3_campaign`, num records: 4625, num bytes: 9.97 MB, allocated bytes: 10.07 MB | State Manager memory usage: Allocated: 9 MB, Used: -1949 bytes, percentage Used -1.859056787650979E-4
2025-12-15 12:35:28 destination INFO pool-6-thread-1 i.a.c.i.d.a.FlushWorkers(printWorkerInfo):127 [ASYNC WORKER INFO] Pool queue size: 0, Active threads: 0
2025-12-15 12:36:28 destination INFO pool-3-thread-1 i.a.c.i.d.a.b.BufferManager(printQueueInfo):94 [ASYNC QUEUE INFO] Global: max: 371.22 MB, allocated: 20.07 MB (20.068470001220703 MB), %% used: 0.054061035443982025 | Queue `ab_3_campaign`, num records: 4625, num bytes: 9.97 MB, allocated bytes: 10.07 MB | State Manager memory usage: Allocated: 9 MB, Used: -1949 bytes, percentage Used -1.859056787650979E-4
2025-12-15 12:36:28 destination INFO pool-6-thread-1 i.a.c.i.d.a.FlushWorkers(printWorkerInfo):127 [ASYNC WORKER INFO] Pool queue size: 0, Active threads: 0
2025-12-15 12:37:28 destination INFO pool-3-thread-1 i.a.c.i.d.a.b.BufferManager(printQueueInfo):94 [ASYNC QUEUE INFO] Global: max: 371.22 MB, allocated: 20.07 MB (20.068470001220703 MB), %% used: 0.054061035443982025 | Queue `ab_3_campaign`, num records: 4625, num bytes: 9.97 MB, allocated bytes: 10.07 MB | State Manager memory usage: Allocated: 9 MB, Used: -1949 bytes, percentage Used -1.859056787650979E-4
2025-12-15 12:37:28 destination INFO pool-6-thread-1 i.a.c.i.d.a.FlushWorkers(printWorkerInfo):127 [ASYNC WORKER INFO] Pool queue size: 0, Active threads: 0
2025-12-15 12:38:28 destination INFO pool-3-thread-1 i.a.c.i.d.a.b.BufferManager(printQueueInfo):94 [ASYNC QUEUE INFO] Global: max: 371.22 MB, allocated: 20.07 MB (20.068470001220703 MB), %% used: 0.054061035443982025 | Queue `ab_3_campaign`, num records: 4625, num bytes: 9.97 MB, allocated bytes: 10.07 MB | State Manager memory usage: Allocated: 9 MB, Used: -1949 bytes, percentage Used -1.859056787650979E-4
2025-12-15 12:38:28 destination INFO pool-5-thread-1 i.a.c.i.d.a.DetectStreamToFlush(getNextStreamToFlush):109 flushing: trigger info: airbyte_google_ads_backfill - ab_3_campaign, time trigger: true , size trigger: false current threshold b: 50 MB, queue size b: 9.97 MB, penalty b: 0 bytes, after penalty b: 9.97 MB
2025-12-15 12:38:28 destination INFO pool-4-thread-1 i.a.c.i.d.a.FlushWorkers(flush$lambda$6):135 Flush Worker (078ce) -- Worker picked up work.
2025-12-15 12:38:28 destination INFO pool-6-thread-1 i.a.c.i.d.a.FlushWorkers(printWorkerInfo):127 [ASYNC WORKER INFO] Pool queue size: 0, Active threads: 1
2025-12-15 12:38:28 destination INFO pool-4-thread-1 i.a.c.i.d.a.FlushWorkers(flush$lambda$6):141 Flush Worker (078ce) -- Attempting to read from queue namespace: airbyte_google_ads_backfill, stream: ab_3_campaign.
2025-12-15 12:38:28 destination INFO pool-4-thread-1 i.a.c.i.d.a.GlobalMemoryManager(free):78 Freeing 100132 bytes..
2025-12-15 12:38:28 destination INFO pool-4-thread-1 i.a.c.i.d.a.FlushWorkers(flush$lambda$6):159 Flush Worker (078ce) -- Batch contains: 4625 records, 9.97 MB bytes.
2025-12-15 12:38:28 destination INFO pool-4-thread-1 i.a.c.i.d.r.BaseSerializedBuffer(flush):166 Finished writing data to 52e047c0-b2af-4dab-84ef-7b0e452055bc15175004950737430124.csv.gz (389 KB)
2025-12-15 12:38:28 destination INFO pool-4-thread-1 i.a.c.i.d.s.o.StagingStreamOperations(writeRecordsImpl):54 Buffer flush complete for stream ab_3_campaign (389 KB) to staging
Copy code
2025-12-15 12:38:28 destination INFO pool-4-thread-1 i.a.i.d.r.o.RedshiftStagingStorageOperation(writeToStage):113 Uploading records to for airbyte_internal.airbyte_google_ads_backfill_raw__stream_ab_3_campaign to path redshift-staging/airbyte_google_ads_backfill_raw__stream_ab_3_campaign/2025_12_15_11_4a9f4968-6100-428a-9860-6cffc2ae8cc4/
2025-12-15 12:38:28 destination INFO pool-4-thread-1 a.m.s.StreamTransferManager(getMultiPartOutputStreams):329 Initiated multipart upload to creativefabrica-airbyte-live/redshift-staging/airbyte_google_ads_backfill_raw__stream_ab_3_campaign/2025_12_15_11_4a9f4968-6100-428a-9860-6cffc2ae8cc4/60.csv.gz with full ID d2kh9QN8iLwSD4bKxytpQThqXJFuKttarLrN7_OQ7z4tMXzLpJYiE0N61cQ7W69RuSNhy8M_P1F6LEkTgoDNoFLeiAKuQHog9Vgji7J2SNLVN_uVcf_9RwEYjpdYeSaa
2025-12-15 12:38:28 destination INFO pool-4-thread-1 a.m.s.MultiPartOutputStream(close):158 Called close() on [MultipartOutputStream for parts 1 - 10000]
2025-12-15 12:38:28 destination INFO pool-4-thread-1 a.m.s.StreamTransferManager(complete):367 [Manager uploading to creativefabrica-airbyte-live/redshift-staging/airbyte_google_ads_backfill_raw__stream_ab_3_campaign/2025_12_15_11_4a9f4968-6100-428a-9860-6cffc2ae8cc4/60.csv.gz with id d2kh9QN8i...YjpdYeSaa]: Uploading leftover stream [Part number 1 containing 0.38 MB]
2025-12-15 12:38:28 destination INFO pool-4-thread-1 a.m.s.StreamTransferManager(uploadStreamPart):560 [Manager uploading to creativefabrica-airbyte-live/redshift-staging/airbyte_google_ads_backfill_raw__stream_ab_3_campaign/2025_12_15_11_4a9f4968-6100-428a-9860-6cffc2ae8cc4/60.csv.gz with id d2kh9QN8i...YjpdYeSaa]: Finished uploading [Part number 1 containing 0.38 MB]
2025-12-15 12:38:28 destination INFO pool-4-thread-1 a.m.s.StreamTransferManager(complete):397 [Manager uploading to creativefabrica-airbyte-live/redshift-staging/airbyte_google_ads_backfill_raw__stream_ab_3_campaign/2025_12_15_11_4a9f4968-6100-428a-9860-6cffc2ae8cc4/60.csv.gz with id d2kh9QN8i...YjpdYeSaa]: Completed
2025-12-15 12:38:29 destination INFO pool-4-thread-1 i.a.c.i.d.s.S3StorageOperations(loadDataIntoBucket):255 Uploaded buffer file to storage: 52e047c0-b2af-4dab-84ef-7b0e452055bc15175004950737430124.csv.gz -> redshift-staging/airbyte_google_ads_backfill_raw__stream_ab_3_campaign/2025_12_15_11_4a9f4968-6100-428a-9860-6cffc2ae8cc4/60.csv.gz (filename: 60.csv.gz)
2025-12-15 12:38:29 destination INFO pool-4-thread-1 i.a.c.i.d.s.S3StorageOperations(uploadRecordsToBucket):135 Successfully loaded records to stage redshift-staging/airbyte_google_ads_backfill_raw__stream_ab_3_campaign/2025_12_15_11_4a9f4968-6100-428a-9860-6cffc2ae8cc4/ with 0 re-attempt(s)
2025-12-15 12:38:29 destination INFO pool-4-thread-1 i.a.i.d.r.o.RedshiftStagingStorageOperation(writeToStage):124 Starting copy to target table from stage: airbyte_google_ads_backfill_raw__stream_ab_3_campaign in destination from stage: redshift-staging/airbyte_google_ads_backfill_raw__stream_ab_3_campaign/2025_12_15_11_4a9f4968-6100-428a-9860-6cffc2ae8cc4//60.csv.gz.
2025-12-15 12:38:32 destination INFO pool-4-thread-1 i.a.i.d.r.t.RedshiftDestinationHandler(execute):148 Sql b43d2370-27e1-4564-9b0d-5dd030f215ed-fbeb3932-03b1-42b6-8e44-d60ab2a1e4f8 completed in 3939 ms
2025-12-15 12:38:32 destination INFO pool-4-thread-1 i.a.i.d.r.o.RedshiftStagingStorageOperation(writeToStage):136 Copy to target table airbyte_internal.airbyte_google_ads_backfill_raw__stream_ab_3_campaign in destination complete.
2025-12-15 12:38:32 destination INFO pool-4-thread-1 i.a.c.i.d.r.FileBuffer(deleteFile):75 Deleting tempFile data 52e047c0-b2af-4dab-84ef-7b0e452055bc15175004950737430124.csv.gz
2025-12-15 12:38:32 destination INFO pool-4-thread-1 i.a.c.i.d.a.s.GlobalAsyncStateManager(flushStates):153 Flushing states
2025-12-15 12:38:32 destination INFO pool-4-thread-1 i.a.c.i.d.a.s.GlobalAsyncStateManager(flushStates):207 Flushing states complete
2025-12-15 12:38:32 destination INFO pool-4-thread-1 i.a.c.i.d.a.GlobalMemoryManager(free):78 Freeing 0 bytes..
2025-12-15 12:38:32 destination INFO pool-4-thread-1 i.a.c.i.d.a.GlobalMemoryManager(free):78 Freeing 10459373 bytes..
2025-12-15 12:38:32 destination INFO pool-4-thread-1 i.a.c.i.d.a.FlushWorkers(flush$lambda$6):170 Flush Worker (078ce) -- Worker finished flushing. Current queue size: 0
2025-12-15 12:39:28 destination INFO pool-3-thread-1 i.a.c.i.d.a.b.BufferManager(printQueueInfo):94 [ASYNC QUEUE INFO] Global: max: 371.22 MB, allocated: 10 MB (9.998141288757324 MB), %% used: 0.02693328741815257 | Queue `ab_3_campaign`, num records: 0, num bytes: 0 bytes, allocated bytes: 0 bytes | State Manager memory usage: Allocated: 9 MB, Used: -1949 bytes, percentage Used -1.859056787650979E-4
2025-12-15 12:39:28 destination INFO pool-6-thread-1 i.a.c.i.d.a.FlushWorkers(printWorkerInfo):127 [ASYNC WORKER INFO] Pool queue size: 0, Active threads: 0
2025-12-15 12:40:28 destination INFO pool-3-thread-1 i.a.c.i.d.a.b.BufferManager(printQueueInfo):94 [ASYNC QUEUE INFO] Global: max: 371.22 MB, allocated: 10 MB (9.998141288757324 MB), %% used: 0.02693328741815257 | Queue `ab_3_campaign`, num records: 0, num bytes: 0 bytes, allocated bytes: 0 bytes | State Manager memory usage: Allocated: 9 MB, Used: -1949 bytes, percentage Used -1.859056787650979E-4
2025-12-15 12:40:28 destination INFO pool-6-thread-1 i.a.c.i.d.a.FlushWorkers(printWorkerInfo):127 [ASYNC WORKER INFO] Pool queue size: 0, Active threads: 0
2025-12-15 12:41:28 destination INFO pool-3-thread-1 i.a.c.i.d.a.b.BufferManager(printQueueInfo):94 [ASYNC QUEUE INFO] Global: max: 371.22 MB, allocated: 10 MB (9.998141288757324 MB), %% used: 0.02693328741815257 | Queue `ab_3_campaign`, num records: 0, num bytes: 0 bytes, allocated bytes: 0 bytes | State Manager memory usage: Allocated: 9 MB, Used: -1949 bytes, percentage Used -1.859056787650979E-4
2025-12-15 12:41:28 destination INFO pool-6-thread-1 i.a.c.i.d.a.FlushWorkers(printWorkerInfo):127 [ASYNC WORKER INFO] Pool queue size: 0, Active threads: 0
2025-12-15 12:42:28 destination INFO pool-3-thread-1 i.a.c.i.d.a.b.BufferManager(printQueueInfo):94 [ASYNC QUEUE INFO] Global: max: 371.22 MB, allocated: 10 MB (9.998141288757324 MB), %% used: 0.02693328741815257 | Queue `ab_3_campaign`, num records: 0, num bytes: 0 bytes, allocated bytes: 0 bytes | State Manager memory usage: Allocated: 9 MB, Used: -1949 bytes, percentage Used -1.859056787650979E-4
2025-12-15 12:42:28 destination INFO pool-6-thread-1 i.a.c.i.d.a.FlushWorkers(printWorkerInfo):127 [ASYNC WORKER INFO] Pool queue size: 0, Active threads: 0
2025-12-15 12:43:28 destination INFO pool-3-thread-1 i.a.c.i.d.a.b.BufferManager(printQueueInfo):94 [ASYNC QUEUE INFO] Global: max: 371.22 MB, allocated: 10 MB (9.998141288757324 MB), %% used: 0.02693328741815257 | Queue `ab_3_campaign`, num records: 0, num bytes: 0 bytes, allocated bytes: 0 bytes | State Manager memory usage: Allocated: 9 MB, Used: -1949 bytes, percentage Used -1.859056787650979E-4
2025-12-15 12:43:28 destination INFO pool-5-thread-1 i.a.c.i.d.a.DetectStreamToFlush(getNextStreamToFlush):109 flushing: trigger info: airbyte_google_ads_backfill - ab_3_campaign, time trigger: true , size trigger: false current threshold b: 50 MB, queue size b: 0 bytes, penalty b: 0 bytes, after penalty b: 0 bytes
2025-12-15 12:43:28 destination INFO pool-4-thread-2 i.a.c.i.d.a.FlushWorkers(flush$lambda$6):135 Flush Worker (92db3) -- Worker picked up work.
2025-12-15 12:43:28 destination INFO pool-6-thread-1 i.a.c.i.d.a.FlushWorkers(printWorkerInfo):127 [ASYNC WORKER INFO] Pool queue size: 0, Active threads: 1
2025-12-15 12:43:28 destination INFO pool-4-thread-2 i.a.c.i.d.a.FlushWorkers(flush$lambda$6):141 Flush Worker (92db3) -- Attempting to read from queue namespace: airbyte_google_ads_backfill, stream: ab_3_campaign.
2025-12-15 12:43:28 destination INFO pool-4-thread-2 i.a.c.i.d.a.GlobalMemoryManager(free):78 Freeing 0 bytes..
2025-12-15 12:43:28 destination INFO pool-4-thread-2 i.a.c.i.d.a.FlushWorkers(flush$lambda$6):159 Flush Worker (92db3) -- Batch contains: 0 records, 0 bytes bytes.
2025-12-15 12:43:28 destination WARN pool-4-thread-2 i.a.c.i.d.s.c.CsvSerializedBuffer(flushWriter):97 Trying to flush but no printer is initialized.
2025-12-15 12:43:28 destination WARN pool-4-thread-2 i.a.c.i.d.s.c.CsvSerializedBuffer(closeWriter):108 Trying to close but no printer is initialized.
I also see this pattern in the google console API graph:
So around 12:33 it stopped loading (which is the same time the last records are reported in the log), and then it did a burst after and stopped completely after that
7 Views
Open in Slack
PreviousNext
Powered by