VERY URGENT CRITICAL :warning: Hi, I was testing...
# feedback-and-requests
b
VERY URGENT CRITICAL ⚠️ Hi, I was testing a mysql to Bigquery using CDC in airbyte. I put a table for CDC and configured it as Incremental model. When I ran the pipeline for the first time, it gets the snapshot of the table good. But, for the subsequent runs the new records where not captured. I used both Bigquery and bigquery-denormalised destinations. And the behaviour is same for both. Earlier when I was using the older version of airbyte, it was fine. But now, this issue is not solvable for me. Here is the log, in which the pipeline run successfuly, but failed to capture the new records.
u
Copy code
2021-10-06 01:49:47 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-10-06 01:49:47 [32mINFO[m i.a.i.s.m.DebeziumRecordPublisher(lambda$start$1):103 - {} - Debezium engine shutdown.
2021-10-06 01:49:47 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-10-06 01:49:47 [32mINFO[m i.a.i.s.m.DebeziumRecordIterator(computeNext):95 - {} - no record found. polling again.
@Boopathy Raja when you run the first time did the initial load ingest all the data? and subsequent run didnt capture the new records?
z
@[DEPRECATED] Marcos Marx Exactly sir. In the initial run, it got all the rows got ingested and loaded. Then I inserted a new row in the source and ran the pipeline, but no rows got ingested.
u
@Boopathy Raja what version of Airbyte and MySQL connector you were before starting the problem? Could you revert the mySQL to that version?
z
I haven't changed the version of mySQL 5.7, the airbyte alone I have upgraded from 0.27 to 0.29.2
u
this is very weird @Boopathy Raja. Can you open a github issue to the team investigate further? Also, is it possible to share your mysql conf about the CDC params?
u
Sure.. !
u
I think there is a similar kind of issue already reported.. https://github.com/airbytehq/airbyte/issues/2599
u
Can we reopen this issue? Or do I need to generate a new issue?
u
Hey Boopathy can you create a new issue and share so that we can track this