I'm using a custom connector to extract records from the courier list_message and message api, and I am getting the following error from airbyte. What does it mean? 'message='Airbyte could not track the sync progress. Sync process exited without reporting status.', type='io.airbyte.workers.exception.WorkloadMonitorException', nonRetryable=false'

The error message you're seeing,

'Airbyte could not track the sync progress. Sync process exited without reporting status.'

, indicates that the sync process terminated unexpectedly without providing any status updates. This could be due to a variety of reasons such as a bug in the connector, a configuration error, or a system error. In Airbyte, syncs can fail for multiple reasons, and the connector can provide meaningful information about what to do next through

AirbyteErrorTraceMessage

([source](https://docs.airbyte.com/understanding-airbyte/airbyte-protocol#airbyteerrortracemessage)). However, in your case, it seems like the sync process exited without providing this information. To diagnose the issue, you can check the sync logs for more details. If the sync process was cancelled, it might be due to handling large amounts of data or due to rate limits ([source](https://docs.airbyte.com/cloud/managing-airbyte-cloud/review-connection-status#connection-troubleshooting)). If the sync process failed, the error message should help diagnose the next steps. If you're still having trouble figuring out the issue, you might want to consider updating to the latest version of Airbyte, as there have been many platform improvements ([source](https://discuss.airbyte.io/t/2921)). If the issue persists, I recommend reaching out to the Airbyte community or support for further assistance.

@kapa.ai this is the last message in the log, what does it indicate is happening?

2024-10-23 08:10:43 platform > Retry State: RetryManager(completeFailureBackoffPolicy=BackoffPolicy(minInterval=PT10S, maxInterval=PT30M, base=3), partialFailureBackoffPolicy=null, successiveCompleteFailureLimit=5, totalCompleteFailureLimit=5, successivePartialFailureLimit=1000, totalPartialFailureLimit=20, successiveCompleteFailures=0, totalCompleteFailures=0, successivePartialFailures=1, totalPartialFailures=1)
 Backoff before next attempt: 0 seconds

The log message you provided is related to Airbyte's retry mechanism. When a job fails, Airbyte will attempt to retry the job based on certain rules. Each of these sub-invocations of a job is called an attempt. The

RetryManager

is showing the current state of retries. The

completeFailureBackoffPolicy

indicates the backoff policy for complete failures, with a minimum interval of 10 seconds and a maximum of 30 minutes. The

successiveCompleteFailureLimit

and

totalCompleteFailureLimit

are both set to 5, meaning that after 5 successive or total complete failures, the job will stop retrying. The

successivePartialFailureLimit

and

totalPartialFailureLimit

are set to 1000 and 20 respectively, which are the limits for partial failures. In your specific log message,

successiveCompleteFailures

and

totalCompleteFailures

are both 0, indicating that there have been no complete failures yet.

successivePartialFailures

and

totalPartialFailures

are both 1, indicating that there has been one partial failure. The

Backoff before next attempt: 0 seconds

means that the next retry attempt will happen immediately without any delay. For more details, you can refer to the [Airbyte documentation on jobs](https://docs.airbyte.com/understanding-airbyte/jobs#sync-jobs).

@kapa.ai can you point me towards the documentation specifically on the retry manager in the connection builder?

A message from kapa.ai

I'm sorry, but the provided knowledge sources do not contain specific documentation on the Retry Manager in the Connection Builder. However, you can find information about error handling and retry strategies in the Connector Builder UI in the [Airbyte documentation](https://docs.airbyte.com/connector-development/connector-builder-ui/error-handling). This includes details on how to set up backoff strategies and response filters to handle failed requests.