Hey team, I found a strange problem. I use flink SQL to do ETL. However sometimes I need to change SQLs dur to requirement change. I found that when I delete one SQL from the SQL queue or change the SQL sequence, the checkpoint still resume data ETL to perform the old SQL sequence. For example, originally I design SQL with four operators in sequence 1->2->3->4. 1 to 4 are the operator ids.Then I delete operator 2. I want flink to perform operators 1->3->4 in sequence. But in fact flink resumed by checkpoint to still perform 1->2->3->4. The checkpoint seems still remember the old SQL queue by ids and perform it regardless of new SQLs. The SQL changed, I want flink to know the new sequence, the SQLs with new ids assigned automatically. But it seems that flink did not assign new ids for them or new ids did not take effect. I think this is because I could not assigin name and id for SQL operators manually in Flink SQL. It seems that Flink SQL does not provide a way for user to assign operator's name and id manually. If I design ETL with datastream API this should not happen. So is this the defect of Flink SQL?

No, it’s by design

So you mean this is a defect of Flink and it will be solved in the future. Right? You mentioned how to update Flink versions. When you fix it in the future. I could upgrade and solve it.

Not all situations of course

The checkpoint became imcompatible. This is acceptable and considerable. But as I did not use kafka, I want a solution to perform consistance. If I do not use checkpoint to perform it, what chould I use？How can I get a proper solution? I cound not rely on checkpoint now.

Like I said, it’s a deliberate and documented thing. If you can make a contribution to add checkpoint compatibility for SQL and Table API, everyone in the community benefits of that

Thanks! I get the points.