@kapa.ai getting error for some streams in cokroach db Source process exited with non-zero exit code 3

I'm sorry, but the knowledge sources provided do not contain specific information about a CockroachDB source connector or the specific error "Source process exited with non-zero exit code 3" in that context. What I can share based on the available knowledge sources is that "Source process exited with non-zero exit code" errors are a general class of failures in Airbyte, and the exit code can indicate different things: • Exit code 1 – A general source connector error (seen across many connectors). [source error; AlloyDB CDC issue] • Exit code 137 – Typically indicates an out-of-memory (OOM) kill. [Google Ads issue] • Exit code 143 – Usually means the process was terminated (e.g., job cancelled or timed out). [sync cancellation] Exit code 3 is not covered in the available knowledge sources, so I cannot provide a specific explanation for it. General troubleshooting steps you can try: 1. Check the full sync logs for more detailed error messages above the exit code line — the root cause is usually logged before the process exits. 2. Ensure your CockroachDB source connector is up to date. 3. If only some streams are failing, check if those streams have schema or data type issues. 4. Consider reaching out on the Airbyte community forum or filing a GitHub issue with your full logs for more targeted help. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

Malformed non-Airbyte record (connectionId = 99383253-518e-4e84-a03f-fe68f85ad938): Terminating due to java.lang.OutOfMemoryError: Java heap space @kapa.ai

The error

Terminating due to java.lang.OutOfMemoryError: Java heap space

is a well-known issue in Airbyte, particularly when syncing large tables or large volumes of CDC changes. Here's what the knowledge sources tell us: What's Happening The Java process running your source connector is running out of heap memory. This can happen when: • Syncing very large tables (hundreds of millions of rows) • Processing large CDC change sets (e.g., 70M+ row changes in a single run) [Java heapspace issue] • The Postgres source connector's JDBC

fetchSize

is calculated as 60% of JVM heap, making memory requirements unpredictable, especially with variable row sizes (e.g., JSON columns) [Postgres fetchSize OOM] How to Fix It 1. Increase Resource Limits via Helm (values.yaml)

global:
  jobs:
    resources:
      limits:
        cpu: "4"
        memory: "8Gi"
      requests:
        cpu: "2"
        memory: "4Gi"

Or use environment variables:

JOB_MAIN_CONTAINER_CPU_LIMIT

,

JOB_MAIN_CONTAINER_MEMORY_LIMIT

, etc. [SAP HANA issue] 2. Set Connection-Level Resource Requirements You can update the

connection

table directly in the Airbyte database:

UPDATE connection 
SET resource_requirements = '{"cpu_limit": "1.5", "cpu_request": "1.0", "memory_limit": "20Gi", "memory_request": "13Gi"}' 
WHERE id = '<your-connection-id>';

[Java heapspace issue] 3. Known Limitation: Orchestrator Memory Be aware that there is an open issue where

CONTAINER_ORCHESTRATOR_MEMORY_LIMIT

and

REPLICATION_ORCHESTRATOR_MEMORY_LIMIT

environment variables are ignored and the orchestrator always uses 1Gi. The

connection.resource_requirements

workaround works for source/destination containers but not for the orchestrator. [Orchestrator memory issue] 4. Linux Kernel Consideration If you're running on a newer Red Hat-based or Ubuntu Linux kernel (e.g., 6.14.x), there is an open issue where OOM kills are significantly worse. Downgrading the kernel (e.g., to 6.8.x on Ubuntu) has been reported as a workaround. [OOM on newer kernels] In summary, the most actionable steps are to increase memory limits at the connection or global job level, and ensure your connector is up to date. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

@kapa.ai it mostly failed for tables wit size 5.13 3.94 3.25 2.98 1.79 1.52 1.35 1.27 in gb

Based on your previous question about the

OutOfMemoryError: Java heap space

error, the table sizes you're seeing failures on (ranging from 1.27 GB to 5.13 GB) are consistent with what the knowledge sources describe. Why These Sizes Cause OOM The source connector reads up to 10,000 records into memory at a time. For tables with large average row sizes, this can require significant RAM. For example: • A table with an average row size of 0.5 MB will require

0.5 × 10,000 / 1,000 = 5 GB

of RAM [Scaling Airbyte] Additionally, for JDBC-based connectors (like CockroachDB, which uses a Postgres-compatible driver), the

fetchSize

is calculated as 60% of JVM heap, and since it only samples the first 10 rows to estimate row size, it can significantly underestimate memory needs for tables with variable-length columns (TEXT, JSON, arrays). [Postgres fetchSize OOM] Java connectors also default to using only 1/4 of the host's allocated memory for the JVM heap, so even if your pod has 8 GB, the connector may only use 2 GB before OOMing. [MySQL to ClickHouse] Recommended Actions 1. Increase memory limits at the connection level:

UPDATE connection 
SET resource_requirements = '{"memory_limit": "20Gi", "memory_request": "8Gi"}' 
WHERE id = '<your-connection-id>';

[Java heapspace issue] 2. Note the orchestrator limitation: Even with increased source/destination memory, the container orchestrator may still be capped at 1 Gi due to an open bug where

CONTAINER_ORCHESTRATOR_MEMORY_LIMIT

is ignored. [Orchestrator memory issue] 3. General recommendation: Start with a mid-sized instance (4–8 cores) and gradually tune upward based on your workload. [Scaling Airbyte] Given your table sizes (up to ~5 GB), you likely need significantly more than the default memory allocation to handle the in-memory buffering safely. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

@kapa.ai @kapa.ai "internalMessage" : "'GET' request to 'https://gmail.googleapis.com/gmail/v1/users/me/threads/1933ad640d08ee72' failed with status code '403' and error message: 'Quota exceeded for quota metric 'Queries' and limit 'Queries per minute per user' of service 'gmail.googleapis.com' for consumer 'project_number:37329956'.'. Request (body): 'None'. Response (body): '{'error': {'code': 403, 'message': \"Quota exceeded for quota metric 'Queries' and limit 'Queries per minute per user' of service 'gmail.googleapis.com' for consumer 'project_number:37329956'.\", 'errors': [{'message': \"Quota exceeded for quota metric 'Queries' and limit 'Queries per minute per user' of service 'gmail.googleapis.com' for consumer 'project_number:37329956'.\", 'domain': 'usageLimits', 'reason': 'rateLimitExceeded'}], 'status': 'PERMISSION_DENIED', 'details': [{'@type': 'type.googleapis.com/google.rpc.ErrorInfo', 'reason': 'RATE_LIMIT_EXCEEDED', 'domain': 'googleapis.com', 'metadata': {'quota_unit': '1/min/{project}/{user}', 'quota_location': 'global', 'service': 'gmail.googleapis.com', 'consumer': 'projects/37329956', 'quota_limit_value': '15000', 'quota_metric': 'gmail.googleapis.com/default', 'quota_limit': 'defaultPerMinutePerUser'}}, {'@type': 'type.googleapis.com/google.rpc.Help', 'links': [{'description': 'Request a higher quota limit.', 'url': 'https://cloud.google.com/docs/quotas/help/request_increase'}]}]}}'. Response (headers): '{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'content-encoding': 'gzip', 'date': 'Wed, 25 Mar 2026 134733 GMT', 'server': 'ESF', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'Alt-Svc': 'h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000', 'Transfer-Encoding': 'chunked'}'.", "externalMessage" : "HTTP Status Code: 403. Error: Forbidden. You don't have permission to access this resource.", "metadata" : { "attemptNumber" : 1,

The error you're seeing is a Gmail API rate limit error: >

Quota exceeded for quota metric 'Queries' and limit 'Queries per minute per user'

with a limit of 15,000 queries per minute per user. This is happening because the Gmail source connector is making too many requests to the Gmail API in a short period, exceeding your Google Cloud project's quota. What's Happening The Gmail connector is hitting the

defaultPerMinutePerUser

quota limit (15,000 queries/min/user) on the Gmail API. Google returns a

403

status code for this rate limit error, which Airbyte's default error handler may treat as a non-retryable permission error rather than a rate limit. [Error Handling] How to Fix It 1. Request a Quota Increase from Google The error response itself suggests this: go to Google Cloud Console and request a higher quota for the Gmail API on your project (

project_number:37329956

). 2. Configure Rate Limiting in the Connector (if using a custom/forked connector) If you're building or modifying a connector, you can use Airbyte's HTTP API Budget to proactively limit request rates and avoid hitting the quota. For example, using a `MovingWindowCallRatePolicy`: [Rate limiting]

api_budget:
  type: "HTTPAPIBudget"
  policies:
    - type: "MovingWindowCallRatePolicy"
      rates:
        - limit: 14000   # stay under the 15,000/min limit
          interval: "PT1M"
      matchers:
        - method: "GET"
          url_base: "https://gmail.googleapis.com"
          url_path_pattern: "^/gmail/v1/users/me/threads"

3. Handle 403 as a Rate Limit Error Since Gmail uses

403

for rate limit errors (not the standard

429

), you can configure the connector's error handler to treat

403

as a retryable rate limit response. [Error Handling] 4. Check the Gmail Connector's Rate Limit Headers The HTTP API Budget also supports reading rate limit headers (

X-RateLimit-Reset

,

X-RateLimit-Remaining

) to automatically back off when limits are approaching. [Global Configuration] The most immediate fix is to request a quota increase from Google, as the connector is operating within normal bounds but your project's quota is being exhausted. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

@kapa.ai i am using airbyte open source helm chart, how do i change

It looks like your message got cut off! Could you complete your question? For example, are you asking how to change: • Memory/resource limits for connectors (based on our earlier conversation about OOM errors)? • Rate limiting configuration? • Something else in the Helm chart? Please share the rest of your question and I'll do my best to help! 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

the Rate limiting configuratio