My Mysql streams have the following error after up...
# replication-ask-ai
j
My Mysql streams have the following error after upgrading to Airbyte V2 :
Copy code
Input was fully read, but some streams did not receive a terminal stream status message
k
This is a known bug in Airbyte's Bulk Load CDK (CDK v3), which is used by modern destination connectors. The error occurs when the destination finishes reading all input but one or more streams never received a terminal
STREAM_STATUS
(COMPLETE) message from the source. Root Cause The error originates in
SyncManager.markInputConsumed()
, which enforces that all streams must have received a terminal status before marking input as consumed. When the source doesn't emit that terminal message, the destination throws: [Streams without status message]
Copy code
io.airbyte.cdk.TransientErrorException: Input was fully read, but some streams did not receive 
a terminal stream status message. If the destination did not encounter other errors, this likely 
indicates an error in the source or platform.
This has been observed across multiple source connectors (MySQL, MSSQL, declarative YAML sources, ClickHouse, etc.) when paired with CDK v3 destinations. [MySQL ClassCastException issue] Workarounds 1. Downgrade the destination connector to a version that uses the legacy CDK (pre-CDK v3). For example, BigQuery v2.x instead of v3.x. These older versions do not require terminal stream status messages. [Streams without status message] 2. Check if the source error is the root cause: In some cases (e.g., the MySQL issue), the source itself exits with a non-zero code, which prevents the terminal status from being emitted. Fixing the source-side error may resolve the destination error as well. [MySQL ClassCastException issue] Fix in Progress A draft PR (#74803) has been proposed to address this in the Bulk CDK. The fix modifies
PipelineEventBookkeepingRouter.close()
to unconditionally call
manager.markEndOfStream(sawComplete)
for every stream, rather than only for streams that received an explicit
COMPLETE
message. This would prevent the hard
TransientErrorException
failure, though streams without an explicit COMPLETE would still exit with
StreamsIncompleteException
. [AI-Triage PR] As of the latest information available, this issue remains open and unresolved with no official permanent fix deployed. You may want to monitor GitHub issue #72367 and #74802 for updates, or join the Airbyte Community Slack for peer support. šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
j
@kapa.ai What BigQuery versions will fix my issue ? 3.0.18 ? Or Only 2.x ?
k
Based on the knowledge sources available, I cannot give you a definitive answer about whether BigQuery 3.0.18 specifically fixes the
TransientErrorException
/ "streams without terminal status" issue for MySQL sources. Here's what the knowledge sources do say: • BigQuery 3.0.18 (deps upgrade PR) upgrades BigQuery Cloud dependencies (
google-cloud-bigquery
2.53.0 → 2.62.0) and bumps the CDK version (0.2.0 → 1.0.6). There is no mention of it fixing the terminal stream status issue. • The draft fix for the terminal stream status problem is tracked in PR #74803 (bumping
bulk-cdk-core-load
to 1.0.7), but the knowledge sources do not confirm whether this fix was included in BigQuery 3.0.18 or any other specific 3.x release. [AI-Triage PR] • The confirmed workaround mentioned by community members is downgrading to BigQuery v2.x, which uses the legacy CDK and does not require terminal stream status messages. [Streams without status message] In summary: The knowledge sources do not confirm that BigQuery 3.0.18 resolves your specific issue. The only confirmed workaround is downgrading to BigQuery v2.x. I'd recommend monitoring GitHub issue #72367 and #74802 for an official fix, or reaching out via the Airbyte Community Slack for the latest status. šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
j
@kapa.ai I just downgraded to bigquery to 2.12.4 but I still have the error. Do I need to do anything on my sync to make the change ?
k
The knowledge sources do not contain specific information about steps required after downgrading the BigQuery destination connector to resolve this issue (e.g., whether a connection reset or full clear is needed). However, based on what the knowledge sources do mention: • When rolling back to a previous destination version after a Destinations V2 upgrade, Airbyte's documentation recommends clearing all connections that synced at least once to the previously upgraded destination: [Upgrading to Destinations V2] > "Clear all connections which synced at least once to a previously upgraded destination. To be safe, you may clear all connections sending data to a previously upgraded destination." This is the most relevant guidance available. I'd suggest: 1. After downgrading to BigQuery v2.x, clear your affected connections in the Airbyte UI before re-running the sync. 2. If the error persists even after clearing, it may be worth reaching out via the Airbyte Community Slack or checking GitHub issue #72367 for further guidance, as the knowledge sources don't provide more detail on this specific scenario. šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.