https://slingdata.io logo
I am trying to debug issues with
# sling-help
m
I am trying to debug issues with replication between mysql servers. I have a handful of tables that fail with the error
inserted in temp table but table count (912770) != stream count (912772). Records missing/mismatch. Aborting
. This is using truncate mode with direct_insert: true, not sure how i can get more information about which rows are different, its a pretty simple table with a single primary key.
f
can you try with
target_options.use_bulk: false
and let me know how that works? for loading in MySQL, a bulk loading method was added to make loading much faster. In the tests, things worked fine, even for various white spaces, quotes with escapes. I think something may is not being escaped properly. If you turn off bulk mode, and it works, that would confirm the issue.
m
yeah i noticed the new LOAD INFILE message
trying now
that worked
f
Ok thanks, will try to reproduce. Just use that work-around for now
m
btw the new mode: definition-only option, i was previously using source_options.limit: 1, is that essentially the same (it wouldn't let me do limit: 0)
f
yes, pretty much the same, just more convenient, uses
WHERE 1=0
hey @mitty, when you have a moment, can you try running your job with the
development
channel? I pushed some changes. Please confirm if that works OK with your data. https://cdn.discordapp.com/attachments/1465792591160279170/1465806319310209225/image.png?ex=697a7203&is=69792083&hm=9e187db706f2084e7fbd9ad802d6aafa9305a70c601f8f12738bbcad91522050&
m
@flarco i tested that against the 12 tables we had that were failing and they work now! thanks!
f
great 👍
new release is out, so you don't need to stay on dev channel
3 Views