hey all, I recently enabled stateful ingestion on ...
# troubleshoot
a
hey all, I recently enabled stateful ingestion on the Trino source, today when the job ran our Trino cluster was down, the job was marked as “Failed” in the ingestion UI, but it went ahead and soft-deleted all Trino entities. My expectation would be that things would halt if the job fails and it would not continue to soft delete everything. Is this a known bug? We’re on version 0.9.5 if that’s relevant but I haven’t found anything about this in future version release notes
a
Hi, I would roll back the ingestion run, it should return things to normal
I haven’t seen this reported before as a bug though, could you provide ingestion logs?
a
here’s the log, i scrubbed out db and table names in case anything sensitive leaked in there but structure is intact
an option to not soft-delete if any failures are encountered might solve the issue, I wonder if that would affect the state in future runs though?
a
Hey @abundant-airport-72599, this would be a good feature request https://feature-requests.datahubproject.io. You could also open an RFC to get some discussion around it going. @gentle-hamburger-31302 might be able to speak more to the ingestion behavior you’re seeing