@kapa.ai I am wondering if anyone has had experience with the Google Ads connector and concurrency. It seems like the source connector is still limited by 1 CPU and is taking an extremely long time to import our large sources. I do see that the worker count is being sent and updated in the num_workers of the connectorConfig.json, but I also see that the var

CONCURRENT_SOURCE_STREAM_READ:
false

. I am currently running the latest version of the connector 4.1.6 and running Airbyte OSS version 2.0.1 within a GKE cluster with autopilot. There are 4 cores available to each of the containers under the worker pod for this connection and the destination and orchestrator are barely taxed. I do notice that the connector is written in python, which may limit it to 1CPU, but I am unsure. Could I ask for community help on this? Thank you.

You’re right to look at both

num_workers

and

CONCURRENT_SOURCE_STREAM_READ

here. From the available docs and forum content, there are a few key points: 1. Connector‑level concurrency support (Python CDK) ◦ Parallelism inside a single connector is only available if the connector is implemented using the Python CDK’s concurrent mode (

ConcurrentSourceAdapter

,

ConcurrentSource

, etc.). [Concurrent tutorial] ◦ The concurrency tutorial shows that a connector must: ▪︎ Inherit from

ConcurrentSourceAdapter

▪︎ Initialize a

ConcurrentSource

with a

concurrency_level

(often driven by a

num_workers

config field) ▪︎ Wrap streams with the concurrent adapters. [Concurrent tutorial] The knowledge sources do not state that the Google Ads connector (4.1.6) is using this concurrent CDK pattern, so I cannot confirm that

num_workers

is actually enabling intra‑connector parallelism. 2.

num_workers

in the Google Ads connector config ◦ The Terraform schema for

source_google_ads

exposes

num_workers

with the description: > “The number of concurrent workers to use for syncing. Increasing this value may speed up syncs for accounts with many customers or streams. Adjust based on your API usage and rate limits.” [Google Ads resource] ◦ This shows the intended behavior, but the docs don’t confirm how much parallelism you actually get in practice, nor how it interacts with

CONCURRENT_SOURCE_STREAM_READ

. 3.

CONCURRENT_SOURCE_STREAM_READ

and API‑source concurrency ◦ A forum answer notes that “API sources aren’t parallel” yet, and that concurrency is being rolled out via the Python CDK and tested on specific connectors (Stripe), with low‑code connectors not using concurrent execution yet. [Forum concurrency] ◦ That same thread clarifies: > “Is low code using concurrent execution: no, not yet. We will add it at some point, but no target date yet. We have concurrency support in Python CDK though.” [Forum concurrency] This suggests that even if

CONCURRENT_SOURCE_STREAM_READ

is

false

, you only get real parallelism if the connector is explicitly built on the concurrent CDK. The knowledge sources do not say that Google Ads has been migrated to that pattern. 4. CPU usage and scaling behavior ◦ At the platform level, you can scale how many sync jobs run in parallel via

MAX_SYNC_WORKERS

and related env vars, and by ensuring your cluster can schedule enough pods. [Scaling Airbyte; Scaling guide] ◦ However, that affects parallel jobs, not how many CPUs a single connector process can effectively use. The docs do not provide any tuning knob to force a single Google Ads sync to use more than one CPU core. 5. Current limitations / uncertainty ◦ The knowledge sources do not explicitly document: ▪︎ Whether

source-google-ads

4.1.6 is implemented with

ConcurrentSourceAdapter

. ▪︎ Whether

CONCURRENT_SOURCE_STREAM_READ=false

is expected for this connector or how to override it. ▪︎ Any confirmed behavior about the connector being effectively single‑threaded / single‑CPU in practice. Given that, based only on the provided sources: • It’s plausible that your Google Ads sync is effectively single‑threaded and not taking advantage of the 4 cores available in the worker pod. •

num_workers

is exposed and intended to control concurrency, but it may not translate into significant parallelism if the connector isn’t fully wired to the concurrent CDK. • Platform‑level scaling (more workers, more concurrent syncs) won’t speed up a single large Google Ads sync; it only helps if you can split work into multiple connections or jobs. What you can do within the documented behavior Within what’s documented: 1. Increase parallel jobs (if you can split work): ◦ If you can logically split your Google Ads workload into multiple connections (e.g., by customer IDs or subsets of streams), you can run them in parallel by: ▪︎ Increasing

MAX_SYNC_WORKERS

(and related worker env vars) on the worker. [Scaling Airbyte] ▪︎ Ensuring your GKE cluster can schedule enough pods. [Scaling guide] 2. Use

num_workers

carefully: ◦ Keep

num_workers

at a moderate value (the default is 3) and adjust based on Google Ads API rate limits and observed behavior, as per the connector schema description. [Google Ads resource] 3. If you’re open to custom development: ◦ The only documented way to get true intra‑connector concurrency is to implement or extend the connector using the Python CDK’s concurrent mode (

ConcurrentSourceAdapter

,

ConcurrentSource

, etc.). [Concurrent tutorial] ◦ The docs don’t show that this has already been done for Google Ads. Because the knowledge sources don’t explicitly confirm the internal implementation of

source-google-ads

4.1.6, I can’t definitively say whether what you’re seeing is a bug or just the current design. From the docs and forum content, it is consistent with the current limitations of API‑source concurrency. If you need a definitive answer on whether Google Ads 4.1.6 is using the concurrent CDK and how

CONCURRENT_SOURCE_STREAM_READ

is set for that connector, the next step would be to open or follow an issue in the Airbyte GitHub repo (there is already at least one open issue about Google Ads performance/timeouts). [Google Ads issue] 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.