Hi,We created the flink cep Streaming Application,...
# troubleshooting
a
Hi,We created the flink cep Streaming Application,it was running successfully with Multiple Pattern we have added,We stopped the Job with SavePoint,Removed one Pattern from the job Now when restarting it using savepoint it is giving exception: org.apache.flink.util.FlinkRuntimeException: State newround_1:157 does not exist in the NFA. NFA has states [Final State $endState$ [ ]), Start State login_0 [ StateTransition(TAKE, from login_0 to $endState$, with condition), ])] at org.apache.flink.cep.nfa.NFA.isStartState(NFA.java:162) at org.apache.flink.cep.nfa.NFA.isStateTimedOut(NFA.java:342) at org.apache.flink.cep.nfa.NFA.advanceTime(NFA.java:294) at org.apache.flink.cep.operator.CepOperator.advanceTime(CepOperator.java:429) at org.apache.flink.cep.operator.CepOperator.onEventTime(CepOperator.java:310) at org.apache.flink.streaming.api.operators.InternalTimerServiceImpl.advanceWatermark(InternalTimerServiceImpl.java:302) at org.apache.flink.streaming.api.operators.InternalTimeServiceManagerImpl.advanceWatermark(InternalTimeServiceManagerImpl.java:180) at org.apache.flink.streaming.api.operators.AbstractStreamOperator.processWatermark(AbstractStreamOperator.java:599) at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitWatermark(OneInputStreamTask.java:239) at org.apache.flink.streaming.runtime.watermarkstatus.StatusWatermarkValve.findAndOutputNewMinWatermarkAcrossAlignedChannels(StatusWatermarkValve.java:200) at org.apache.flink.streaming.runtime.watermarkstatus.StatusWatermarkValve.inputWatermark(StatusWatermarkValve.java:105) at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:136) at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:105) at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65) at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:542) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231) at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:831) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:780) at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:935) at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:914) at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:728) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:550) at java.lang.Thread.run(Thread.java:748)
m
That’s because you’ve changed the operation with a changed pattern
So the savepoint is incompatible
a
but i am using the AllowedNonRestoredState option that allows a job to start from a state snapshot (a savepoint or checkpoint) that contains state that has nowhere to be restored to in the job being started. In other words, some state was dropped.
And Also when i removed the existing operator from the running job it is giving the same error but adding the new operator it works fine using savepoint restart
m
In other words, some state was dropped.
Only if the UUIDs are set for all operators, as outlined on https://nightlies.apache.org/flink/flink-docs-master/docs/ops/production_ready/#set-uuids-for-all-operators For CEP, you can't set UUIDs.
a
ok thanks for the info So we cannot remove any CEP operator from the Job or is there any way available?
m
There's no workaround for this
a
is there any alternative of CEP?
m
No
MATCH_RECOGNIZE in Flink SQL also uses CEP under the hood
a
So we cannot use dynamic rules in flink using any lib ?
m
Correct. There was a proposal in FLIP-200 but that has stalled, see https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=195730308