Hi, i'm looking for some guidance on how to conver...
# troubleshooting
f
Hi, i'm looking for some guidance on how to convert our FlinkSQL to Datastream API and ensure that we continue processing from where the SQL left off Our SQL is luckily very simple. It reads from an S3 bucket ( or folder ), does some basic casting of the columns and emits to kafka What we are not sure of when migrating it to the Datastream API ( using FileSource ), will flink be able to consume the savepoint of the SQL job? When doing the transition, do we have to first entirely stop the SQL job & then start the Datastream job? Thanks! ( i'm fairly new to Flink btw, sorry for any terminology mishaps )
t
It might be easiest to manually read the contents of the save point (either with the state processor API or by directly deserialising it, I've seen some code floating around for the latter, but I don't recall where)
And then start your new task with starting state/offset based on what you read from the savepoint
t
Yes. At least that's what we ended up doing in some similar situations.
This random Korean post is the only published example of bypassing the state processor API that I am aware of https://ngela.tistory.com/49
πŸ‘€ 1
From what I recall I ended up using the basics from there and then just dig through the API docs for CheckpointMetadata