Hello guys,
I created a custom ingestion source fo...
# troubleshootg
gentle-camera-33498
07/20/2022, 7:11 PMHello guys,
I created a custom ingestion source for Dataset Assertions. The ingestion is ok; the AssertionInfo is on my MySQL database and the AssertionRunEvent on Elasticsearch index. But I can't see the Assertions on "Validation" tab and I can't even delete de metadata. Any idea what could cause this?
Copy code
datahub delete --env PROD --entity_type assertion --hard
This will permanently delete data from DataHub. Do you want to continue? [y/N]: y
[2022-07-20 16:01:44,919] INFO {datahub.cli.delete_cli:234} - datahub configured with <https://------>-
[2022-07-20 16:01:46,660] INFO {datahub.cli.delete_cli:247} - Filter matched 0 entities. Sample: []
This will delete 0 entities. Are you sure? [y/N]: Nplus1 1
s
strong-kite-83354
12/14/2022, 2:56 PMHi Patrick - sorry for the random request elsewhere! Did you resolve this issue? I have a similar problem, I can make Assertions appear on the Validation tab but when I delete the parent datasets I can't re-add the Assertions.
g
gentle-camera-33498
12/15/2022, 12:23 PMThe problem above was a rendering issue that I was having. Upgrading datahub to v0.9.1 solved my situation because of a validation tab rendering bug fix implementation
gentle-camera-33498
12/15/2022, 12:27 PMI suppose that your problem is that you performed a soft delete in your metadata. This produces a metadata aspect called status, used by the frontend to define when metadata should appear on the UI. If you soft delete entities and try to reingest them, you need to ingest the status metadata and set the status value to removed = false.
Do you have access to your datahub database? If yes, try to search for the assertion aspects and see if there is a status aspect. If the status value is removed = true, this is the cause of your issue.
s
strong-kite-83354
12/15/2022, 12:56 PMThanks for the info! I got around this problem by explicitly deleting the assertions immediately before re-ingestion (using the /entities?action=delete endpoint). I guess my problem was therefore that when I (hard) deleted the parent dataset the assertions got their status set to "removed" and this wasn't updated when I reloaded the assertions. I'm using the quickstart docker version of DataHub, although I can find the port number, user name and password I get access denied.
g
gentle-camera-33498
12/15/2022, 6:40 PMThat is weird. Did you docker compose file exposes a port to your database?
Other possibility is to search for the assertions with the GraphQL API.