For source connectors, if an exception occurs duri...
# replication-troubleshooting
w
For source connectors, if an exception occurs during reading records, is it normal for the state to not be stored? I am seeing "Source did not output any state messages" and "State capture: No state retained." in the logs and wanted to confirm this is expected behavior.
n
Yes, that is the expected behavior!
w
So just to confirm: in an initial import, say the first 10,000 records read have no issues but the 10,001 creates an exception. None of the first 10,000 records will be stored and the state will not be updated.
n
My apologies - I misunderstood. There is a checkpointing mechanism that saves the state: Sources are able to emit state in order to allow checkpointing data replication. The goal is that given wherever a sync stops (whether this is due to all data available at the time being replicated or due to a failure), the next time the Source attempts to extract data it can pick up where it left off and not have to go back to the beginning. This concept enables incremental syncs--syncs that only replicate data that is new since the previous sync. State also enables Partial Success. In the case where during a sync there is a failure before all data has been extracted and committed, if all records up to a certain state are committed, then the next time the sync happens, it can start from that state as opposed to going back to the beginning. Partial Success is powerful, because especially in the case of high data volumes and long syncs, being able to pick up from wherever the failure occurred can costly re-syncing of data that has already been replicated. https://docs.airbyte.com/understanding-airbyte/airbyte-protocol/#state--checkpointing Does this answer your question?
w
Is there any way to see the current state from the GUI?
n
No, there is not, unfortunately.
w
What about in the backend default Postgres database?
n
For that I would encourage you to explore the documentation for the API: https://airbyte-public-api-docs.s3.us-east-2.amazonaws.com/rapidoc-api-docs.html#servers