Hi everyone! I have a few ingestion runs I'd like...
# ingestion
b
Hi everyone! I have a few ingestion runs I'd like to rollback and had a couple quick questions. I read through the documentation on rolling back ingestion batch runs and it's pretty easy; I was able to roll back two ingestions last week using these commands. However, it seems to only do a soft delete since the rows are still present in mysql and the
--hard
option does not appear to be compatible with the rollback command. Is there a way to do a hard delete when rolling back an ingestion run? If not, is this something we could expect to see in future releases? Thanks!
h
Hello @bland-balloon-48379 The default rollback behavior is analogous to hard delete. However it might have done some soft deletes, due to presence of some manual modifications to entities ingested during that run. Please read this followup document on rolbacks for more details. To force the hard delete behavior, you can use
--nuke
flag with rollback command, as mentioned in the document.
b
Hey @hundreds-photographer-13496, thanks for the response. I did read that section of the documentation you provided. What is strange is I am certain that none of the items ingested had any sort of modifications done to them, so I would have expected a hard delete to occur for them. However, I still have ~54,000 rows in mysql related to that run. I may try the
--nuke
option you suggested, but am confused why that should even be necessary. It's good to know that a hard delete is the expected behavior though.
I will say that it appears some stuff was deleted because out of the ~54K aspects present from that run, I only have 3 browsePaths, 2 datasetProperties, and 2 schemaMetadata aspects. Other aspects like datasetKey, status, etc. are present in large quantities to varying degrees though. Curious as to why it failed to delete a majority of the rows though. I'll keep digging and update this thread if I discover anything.
Forgot to post this update, but I tried again with the
--nuke
flag and it cleared the items as expected. Still not sure why that flag was needed though since no metadata changes were made to those entities and no rows were flagged as being "unsafe" to delete.
@nice-oil-28310 Tag, you're it!