Hello
How do we change the stateful ingestion commit policy, what are the available options and will there be any consequences related to this change.
Problem: With current commit policy on_no_errors, the checkpoint is not saved even if there is a warning during ingestion. Due to this stale metadata remains present in datahub.
#Stateful Ingestion
# Datahub Version: v0.8.38