Any suggestion?
# troubleshootingm
Mourad HARMIM
06/02/2023, 11:45 AMAny suggestion?
d
David Anderson
06/02/2023, 2:02 PMWhile this is an interesting capability for some use cases, I think there's a consensus that this feature could be more useful if a different approach were taken. However, so far no one has stepped up to drive this forward.
π 1
David Anderson
06/02/2023, 2:03 PMOne currently available alternative is to use the state processor API to read state out of a savepoint.
π€ 1
m
Mourad HARMIM
06/02/2023, 2:22 PMI appreciate your reply.
I'll look up the documentation to learn more about it.
Mourad HARMIM
06/06/2023, 2:36 PMHi ! I checked the state processor api and it matches the need to access the states however :
β’ By essence it's a cold access at the time we lookup the state of a checkpoint, the data might have changed in the running job
β’ It requires to explicily name the operator identifier to access its states
I'll keep it in mind since it surely be useful for an other usecase.
Thank you again π
Mourad HARMIM
06/06/2023, 2:40 PMI thought about some other approaches :
β’ Use a side output to sink the state to a kafka or another external system
β’ Use the gauge metrics to push the values
β’ Or simply log the states and parse them in orde to be easily readable
If you have the time i'd like to know your opinion about these ideas.
Thank you
d
David Anderson
06/07/2023, 10:02 AMSending the state you care about to some queryable external data store is a commonly used approach.
π 1