https://linen.dev logo
Join Slack
Powered by
Hi <@U04S8PLJWMT>, i have a apple search ads sync ...
# replication-ask-ai
s
Hi @kapa.ai, i have a apple search ads sync 
keywords_report_daily
in airbyte have a status of 
scheduled for next sync
while other streams are syncing, I retried multiple times there is no way to get the stream start syncing
k
Based on the knowledge sources, the "Queued for next sync" status means the stream hasn't synced yet and is going to sync in the next scheduled sync ā€” it is not queued for the current ongoing sync. [Connection status] This is expected behavior in Airbyte: only one sync per connection can run at a time, and streams are processed within a sync job. If 
keywords_report_daily
is showing "Queued for next sync" while other streams are actively syncing, it means it was not included in the current sync run. One important thing to be aware of for 
keywords_report_daily
specifically: version 1.0.0 of the Apple Search Ads connector introduced a breaking change for this stream, migrating it from a per-partition state to a global state cursor. If you recently upgraded to 1.0.0, you may need to clear the stream's data and re-trigger a sync. [Apple Ads Migration Guide] Here are the recommended steps to resolve this: 1. Go to Connections in the main nav bar and select the affected connection. 2. Click the Status tab. 3. In the Enabled streams list, find 
keywords_report_daily
, click the three dots on the right, and select Clear Data. 4. Once the clear succeeds, click Sync Now to trigger a fresh sync. If the issue persists after clearing and re-syncing, it may be worth checking whether you're on the latest connector version (1.0.8 as of March 2026) and whether there are any underlying OAuth token expiration issues that could be interrupting the sync. [Apple Ads Changelog] šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
s
I can't sync keywords_report_daily as it is stuck at 
Queued for next sync
@kapa.ai
k
Since you've already tried retrying multiple times without success, here are some additional things to consider: Steps to Try 1. Clear the stream data and re-sync: ā—¦ Go to Connections ā†’ select your connection ā†’ Status tab. ā—¦ Find 
keywords_report_daily
in the enabled streams list, click the three dots, and select Clear Data. ā—¦ Once complete, click Sync Now. [Apple Ads Migration Guide] 2. Check your connector version: If you're on version 1.0.0 or later, the 
keywords_report_daily
stream underwent a breaking change (per-partition state ā†’ global state cursor). Clearing the data as described above is the recommended migration step. [Apple Ads Migration Guide] 3. OAuth token expiration: There is a known issue where the OAuth token (which expires in 3600 seconds) can expire mid-sync for long-running streams like 
keywords_report_daily
, causing failures. This was addressed in version 0.8.0, which added a token refresh endpoint override configuration. Make sure you're on at least version 0.8.0 (latest is 1.0.8). [Apple Ads Changelog] 4. Check for configuration errors: If the connection status page shows a red configuration error, navigate to the source settings, re-test, and save before attempting another sync. [Connection status] If none of these steps resolve the issue, the knowledge sources don't provide further specific guidance ā€” I'd recommend reaching out to Airbyte support for deeper investigation. šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
s
@kapa.ai the connection status is green
k
Since the overall connection status is green (Healthy), it means the most recent sync for the connection succeeded. [Connection status] However, an individual stream like 
keywords_report_daily
can still show "Queued for next sync" even when the connection is healthy ā€” this simply means that particular stream was not included in the last sync run and is scheduled to be picked up in the next one. Given this, here's what I'd suggest: 1. Manually trigger a sync by clicking Sync Now and monitor whether 
keywords_report_daily
moves from "Queued for next sync" to "Syncing" during that run. 2. If it still doesn't sync, try the Clear Data approach: ā—¦ Go to Status tab ā†’ find 
keywords_report_daily
ā†’ click the three dots ā†’ select Clear Data. ā—¦ Then click Sync Now. [Apple Ads Migration Guide] The knowledge sources don't provide further specific guidance on why a stream would persistently remain in "Queued for next sync" when the connection is otherwise healthy. If the issue continues after clearing and re-syncing, I'd recommend reaching out to Airbyte support for a deeper investigation. šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
s
@kapa.ai my data sync seems to be all slower than using airbyte 2.0 , and there are more sync records in each sync is this normal?
k
Based on the knowledge sources, this is actually the opposite of what you'd expect with Airbyte 2.0. The 2.0 release notes specifically highlight faster sync speed as a key improvement: > "When using faster sync speed, Airbyte syncs data 4-6 times faster, on average, while consuming resources more economically." [Airbyte 2.0 Release Notes] So slower syncs after upgrading to 2.0 would be unexpected and worth investigating. A few things that could explain what you're seeing: 1. First sync after upgrade can be slower: There is a known pattern where the first sync after a major upgrade can be significantly slower than subsequent ones, due to things like schema migrations or changes in how batches are processed. This has been observed with other connector upgrades as well. [Snowflake Destination Issue] 2. More records being synced: If you're seeing more records than expected, this could be related to how the sync mode is configured. For example, a Refresh and Retain Records sync keeps old records and merges in new data, which can lead to increased data storage and slower syncs. [Refreshing Your Data] 3. Large version gap upgrade: If you jumped from a much older version directly to 2.0, the upgrade documentation recommends upgrading incrementally through intermediate versions, as a large version gap can cause issues. [Syncs Issue] I'd recommend: ā€¢ Waiting to see if subsequent syncs return to normal speed (the first post-upgrade sync is often the slowest). ā€¢ Checking your sync mode settings to understand why more records are being synced. ā€¢ If the issue persists, reaching out to Airbyte support with your sync logs for further investigation. šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
s
@kapa.ai this also happens in the subsequent runs
k
Since the slower sync speeds and higher record counts are persisting across subsequent runs (not just the first post-upgrade sync), this is worth investigating more carefully. Based on the knowledge sources, here are some possible explanations: Possible Causes 1. Resource constraints: If your deployment is under-provisioned, syncs can run slower. The documentation recommends monitoring CPU and memory usage. If nodes are under high load, scaling up your deployment or adjusting connector resource limits may help. [Scaling Airbyte] You can adjust connector pod resource limits in your `values.yaml`:
Copy code
global:
     jobs:
       resources:
         limits:
           cpu: ## e.g. 250m
           memory: ## e.g. 500m
         requests:
           cpu: ## e.g. 75m
           memory: ## e.g. 150m
2. More records than expected per sync: If you're seeing unexpectedly high record counts on every run, it's worth checking whether your sync mode is set to Refresh and Retain Records, which keeps old records and merges in new data ā€” this leads to increased storage, compute costs, and slower syncs. [Refreshing Your Data] 3. Incremental sync behaving like full refresh: There are known cases where incremental syncs load the same records repeatedly as if doing a full refresh (e.g., a cursor state issue). This has been observed with connectors like Snowflake and MongoDB. [Snowflake Incremental Issue] Beyond these points, the knowledge sources don't provide enough specific detail to diagnose your exact situation. I'd strongly recommend reaching out to Airbyte support with your sync logs so the team can identify the bottleneck. šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
s
@kapa.ai my airbyte-workload-api-server error out and then my airbyte connection job attempt stuck at init state
Copy code
1 node(s) had untolerated taint {<http://autoscaling.gke.io/defrag-candidate|autoscaling.gke.io/defrag-candidate>: 1773313964}, 3 Insufficient cpu, 4 Insufficient memory. preemption: 0/5 nodes are available: 1 Preemption is not helpful for scheduling, 4 No preemption victims found for incoming pod.
k
The error message you're seeing is a Kubernetes scheduling failure ā€” your cluster doesn't have enough CPU or memory resources to schedule the workload pods, which is why your connection job attempts are stuck at the 
init
state. The key parts of the error: ā€¢ 
3 Insufficient cpu
ā€” 3 nodes don't have enough CPU to schedule the pod ā€¢ 
4 Insufficient memory
ā€” 4 nodes don't have enough memory ā€¢ 
autoscaling.gke.io/defrag-candidate
ā€” GKE is trying to defragment nodes, meaning your cluster is fragmented and resources are not efficiently allocated What to Do 1. Scale up your Kubernetes cluster Airbyte's documentation recommends ensuring your cluster can accommodate the number of pods Airbyte creates. As a rule of thumb: > "Make sure the Kubernetes cluster can schedule up to 
2 x <number-of-possible-concurrent-connections>
pods at once." [Scaling Airbyte] Add more nodes or use larger node types to provide sufficient CPU and memory. 2. Reduce connector resource requests If your connector resource requests are set too high, pods may fail to schedule on available nodes. You can lower the instance-wide resource requests in your `values.yaml`: [Configuring Connector Resources]
Copy code
# In values.yaml (global section)
JOB_MAIN_CONTAINER_CPU_REQUEST: "250m"
JOB_MAIN_CONTAINER_CPU_LIMIT: "500m"
JOB_MAIN_CONTAINER_MEMORY_REQUEST: "256Mi"
JOB_MAIN_CONTAINER_MEMORY_LIMIT: "512Mi"
3. Watch out for millicore parsing issues There is a known bug where the workload-launcher fails to parse millicore values (e.g., 
500m
) and falls back to defaults, which can cause pods to request more CPU than intended and remain in 
Pending
state. A reported workaround is to use integer values (e.g., 
"1"
instead of 
"500m"
) for CPU settings. [Workload resource issue] 4. Disk space Also ensure each node has at least 30GB of disk space, as connector images (~300MB each) can accumulate quickly. [Scaling Airbyte] In summary, the root cause is insufficient cluster resources. You'll need to either scale up your GKE cluster or reduce the resource requests for your workload pods. šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
s
@kapa.ai i am using an autopilot
k
Since you're using GKE Autopilot, the scheduling issue is slightly different from standard GKE. In Autopilot, Google manages the nodes for you, but pods can still fail to schedule if: ā€¢ The requested CPU/memory exceeds Autopilot's per-pod limits ā€¢ The cluster hasn't scaled up fast enough to accommodate new pods The knowledge sources don't contain specific guidance for GKE Autopilot. However, based on what is documented, the most relevant action you can take is to reduce the resource requests for your Airbyte connector pods, so that Autopilot can more easily find capacity to schedule them. You can lower the instance-wide resource requests in your `values.yaml`: [Configuring Connector Resources]
Copy code
global:
  jobs:
    resources:
      limits:
        cpu: "1"        # Use integer values, not millicores (e.g. avoid "500m")
        memory: "1Gi"
      requests:
        cpu: "250m"
        memory: "256Mi"
> Note: There is a known bug where the workload-launcher fails to parse millicore values (e.g., 
500m
) for CPU, so using integer values (e.g., 
"1"
) for CPU limits is recommended as a workaround. Also, you can limit the number of concurrent syncs to reduce the total pod pressure on the cluster: [Scaling Airbyte]
Copy code
worker:
  maxSyncWorkers: 3
  maxCheckWorkers: 3
For GKE Autopilot-specific configuration and limits, I'd recommend consulting the GKE Autopilot documentation alongside Airbyte's Configuring Connector Resources guide, as the knowledge sources don't cover Autopilot-specific behavior in detail. šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
s
@kapa.ai Failed to connect to airbyte-workload-api-server-svc.airbyte/34.118.236.254:8001 this got recreated
k
The error 
Failed to connect to airbyte-workload-api-server-svc
indicates that one of the Airbyte services (likely the cron job or worker) cannot reach the workload API server on port 8001/8007. This is a known connectivity issue seen in Kubernetes/Helm deployments. [Helm install error] Common Causes & Fixes 1. Workload API Server pod is not running or not ready Check if the pod is up:
Copy code
kubectl get pods -n airbyte | grep workload-api-server
If it's in a crash loop or pending state (which could be related to your earlier resource scheduling issues on GKE Autopilot), the service will be unreachable. 2. Restart the worker and workload-api-server deployments A simple restart can resolve transient connectivity issues: [HTTP 502 issue]
Copy code
kubectl -n airbyte rollout restart deployment airbyte-workload-api-server
kubectl -n airbyte rollout restart deployment airbyte-worker

kubectl -n airbyte rollout status deployment airbyte-workload-api-server
kubectl -n airbyte rollout status deployment airbyte-worker
(Replace 
airbyte
with your actual namespace if different.) 3. Mismatched authentication tokens If the workload API server is running but connections are being rejected with 401 errors, it may be a token mismatch between the worker and workload-api-server. You can fix this by setting a consistent token across both deployments: [HTTP 502 issue]
Copy code
TOKEN=$(openssl rand -hex 24)
kubectl -n airbyte set env deployment/airbyte-workload-api-server WORKLOAD_API_AUTH_HEADER=Authorization WORKLOAD_API_BEARER_TOKEN=$TOKEN
kubectl -n airbyte set env deployment/airbyte-worker WORKLOAD_API_AUTH_HEADER=Authorization WORKLOAD_API_BEARER_TOKEN=$TOKEN
4. GKE Autopilot pod rescheduling Given your earlier resource scheduling issues on GKE Autopilot, it's possible the workload-api-server pod was rescheduled to a new IP, but the service DNS (
airbyte-workload-api-server-svc
) should still resolve correctly. If the pod keeps getting recreated due to resource pressure, addressing the resource constraints (as discussed earlier) should help stabilize it. The workload API server listens on port 8007 by default. [Values reference] If you're seeing port 8001 in the error, verify your 
WORKLOAD_API_HOST
environment variable is pointing to the correct port. šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
6 Views
Open in Slack
PreviousNext
Powered by