https://linen.dev logo
Join Slack
Powered by
This message was deleted.
# replication-ask-ai
s
This message was deleted.
k
The knowledge sources do not contain enough information to fully diagnose this specific log output, but I can provide some context based on what's available. What the Logs Show This appears to be a MySQL CDC (Change Data Capture) sync using Debezium. Based on the log sequence, here's what's happening: 1. Debezium connects to the MySQL binlog at 
mysql-bin.049391/13662195
. 2. Initial state construction is triggered — Debezium reads one record to build the initial CDC state (
constructInitialDebeziumState
). 3. The engine is intentionally stopped immediately after (
A record is returned, closing the engine since the state is constructed
). 4. Debezium shuts down cleanly — 
Connector 'io.debezium.connector.mysql.MySqlConnector' completed normally.
5. The initial Debezium state is saved, including schema history and binlog position. The 
ERROR
lines you see (e.g., 
threadExecutor is shut down, terminating keepalive thread
) are not actual errors — they are logged at ERROR level by the underlying 
mysql-binlog-connector-java
library but represent expected behavior during a controlled shutdown. This Looks Like Normal Behavior This log pattern is consistent with Airbyte's MySQL CDC connector performing an initial state bootstrap — it briefly connects to the binlog, captures the current position and schema history, then shuts down Debezium. This is a known pattern in the Airbyte MySQL CDC implementation. [MySQL CDC Tutorial] Potential Concerns • The 
StandaloneConfig values
line at the end is cut off — if there are errors after this point, they would be important to review. • If subsequent incremental syncs are failing with deserialization errors (e.g., 
EOFException
, 
MissingTableMapEventException
), those are separate known issues tracked in the Airbyte GitHub. [CDC Deserialization Issue] [Incremental Sync Issue] Could you share what happens after this log snippet, or describe the specific problem you're experiencing? That would help narrow down whether there's an actual issue here. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
r
@kapa.ai
Copy code
2026-06-09 17:26:17 source INFO o.a.k.c.c.AbstractConfig(logAll):376 StandaloneConfig values: 
	access.control.allow.methods = 
	access.control.allow.origin = 
	admin.listeners = null
	auto.include.jmx.reporter = true
	bootstrap.servers = [localhost:9092]
	client.dns.lookup = use_all_dns_ips
	config.providers = []
	connector.client.config.override.policy = All
	header.converter = class org.apache.kafka.connect.storage.SimpleHeaderConverter
	key.converter = class org.apache.kafka.connect.json.JsonConverter
	listeners = [http://:8083]
	metric.reporters = []
	metrics.num.samples = 2
	metrics.recording.level = INFO
	<http://metrics.sample.window.ms|metrics.sample.window.ms> = 30000
	<http://offset.flush.interval.ms|offset.flush.interval.ms> = 1000
	<http://offset.flush.timeout.ms|offset.flush.timeout.ms> = 5000
	offset.storage.file.filename = /tmp/cdc-state-offset11849526809048179616/offset.dat
	plugin.path = null
	response.http.headers.config = 
	rest.advertised.host.name = null
	rest.advertised.listener = null
	rest.advertised.port = null
	rest.extension.classes = []
	ssl.cipher.suites = null
	ssl.client.auth = none
	ssl.enabled.protocols = [TLSv1.2, TLSv1.3]
	ssl.endpoint.identification.algorithm = https
	ssl.engine.factory.class = null
	ssl.key.password = null
	ssl.keymanager.algorithm = SunX509
	ssl.keystore.certificate.chain = null
	ssl.keystore.key = null
	ssl.keystore.location = null
	ssl.keystore.password = null
	ssl.keystore.type = JKS
	ssl.protocol = TLSv1.3
	ssl.provider = null
	ssl.secure.random.implementation = null
	ssl.trustmanager.algorithm = PKIX
	ssl.truststore.certificates = null
	ssl.truststore.location = null
	ssl.truststore.password = null
	ssl.truststore.type = JKS
	<http://task.shutdown.graceful.timeout.ms|task.shutdown.graceful.timeout.ms> = 5000
	topic.creation.enable = true
	topic.tracking.allow.reset = true
	topic.tracking.enable = true
	value.converter = class org.apache.kafka.connect.json.JsonConverter
2026-06-09 17:26:17 source WARN o.a.k.c.r.WorkerConfig(logPluginPathConfigProviderWarning):379 Variables cannot be used in the 'plugin.path' property, since the property is used by plugin scanning before the config providers that replace the variables are initialized. The raw value 'null' was used for plugin scanning, as opposed to the transformed value 'null', and this may cause unexpected results.
2026-06-09 17:26:17 source INFO o.a.k.c.s.FileOffsetBackingStore(start):58 Starting FileOffsetBackingStore with file /tmp/cdc-state-offset11849526809048179616/offset.dat
2026-06-09 17:26:17 source INFO o.a.k.c.c.AbstractConfig(logAll):376 JsonConverterConfig values: 
	converter.type = key
	decimal.format = BASE64
	schemas.cache.size = 1000
	schemas.enable = false
2026-06-09 17:26:17 source INFO o.a.k.c.c.AbstractConfig(logAll):376 JsonConverterConfig values: 
	converter.type = value
	decimal.format = BASE64
	schemas.cache.size = 1000
	schemas.enable = false
k
The knowledge sources do not contain enough information to provide a definitive diagnosis of this specific log snippet. What This Log Shows This portion of the logs shows the second Debezium engine startup — the actual CDC streaming phase — after the initial state was constructed. Specifically: • 
FileOffsetBackingStore
is starting with a new offset file (
/tmp/cdc-state-offset11849526809048179616/offset.dat
), which is where Debezium will persist its binlog position. • JSON converters are initialized for both key and value. • A 
WARN
about 
plugin.path = null
is logged — this is expected and harmless in Airbyte's embedded Debezium setup. The SSL-related fields in 
StandaloneConfig
(e.g., 
ssl.keystore.location = null
, 
ssl.truststore.location = null
) indicate that no SSL is configured at the Kafka Connect worker level, which is normal since this is an embedded/standalone Debezium instance, not a real Kafka cluster. No Errors Visible Here This log segment appears completely normal. The logs are cut off again — could you share what comes after this point? Specifically, any 
ERROR
or 
WARN
messages following the 
FileOffsetBackingStore
startup would be the most useful for diagnosing any actual issue. [CDC Documentation] 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
r
@kapa.ai the job is stuck more than 10 minutes after snowflake target log row below:
Copy code
2026-06-09 17:27:58 destination INFO sync-operations-9 i.a.i.b.d.o.AbstractStreamOperation(prepareStageForNormalSync):125 IMPORT.PD_COST_MODEL_ITEM: non-truncate sync and no temp raw table. Initial raw table status is null.
2026-06-09 17:27:58 destination INFO sync-operations-9 i.a.i.b.d.o.AbstractStreamOperation(prepareFinalTable):247 Final Table exists for stream PD_COST_MODEL_ITEM
k
The knowledge sources do not contain enough specific information about this exact log pattern (
prepareStageForNormalSync
/ 
prepareFinalTable
with "Initial raw table status is null"), so I cannot give a definitive root cause. However, based on what the knowledge sources do cover, here are the most relevant observations: Likely Cause: Destination Back-Pressure / Buffer Issue This pattern — where the source finishes and the destination appears to stall during table preparation — is consistent with a known issue described in the Airbyte community forums for CDC syncs to Snowflake: > "The sudden drop in throughput is because of the back pressure from the destination. The destination is taking a long time to flush data." [Postgres CDC stuck] Recommended Actions 1. Check the Snowflake query history — look for any long-running or queued queries against 
IMPORT.PD_COST_MODEL_ITEM
. A 
COPY INTO
or 
CREATE TABLE
statement may be hanging on the Snowflake side. [MySQL to Snowflake CDC] 2. Increase the Snowflake destination buffer parameter to 
50
 — this was reported to resolve stalling issues in CDC-to-Snowflake syncs: "What solved the issue in our case was to update the buffer param for Snowflake destination to 50." [Postgres CDC stuck] 3. Limit the number of tables per CDC connection to ~30 — if your connection has many streams, this can cause back-pressure. [Postgres CDC stuck] 4. Ensure you are using internal staging on the Snowflake destination, as it is the recommended loading method for large/CDC syncs. [Large MySQL table] Next Steps Could you share: • How many tables/streams are in this connection? • What loading method (internal staging vs. S3) is configured? • Whether there are any active/queued queries visible in Snowflake's query history? That would help narrow down whether this is a Snowflake-side bottleneck or an Airbyte connector issue. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
7 Views
Open in Slack
PreviousNext
Powered by