Hi all,
deployed datahub for the first time, start...
# troubleshootb
brainy-piano-85560
12/11/2022, 11:12 AMHi all,
deployed datahub for the first time, started with the quickstart + ingestion of sample data, and ingested afterwards some real data.
Now I tried to delete all sample data via the CLI. most of it worked fine, but a few can't be deleted (airflow dag + 5 feast feature tables).
For Airflow:
Failed to execute operation
java.lang.UnsupportedOperationException: Aspect and aspect name is required for create and update operations
For Feast:
A very long stacktrace with a 500 code, java.lang.RuntimeException: Unknown aspect status for entity dataPlatform
Tried both soft & hard deletion.
Thanks for the help :)
brainy-piano-85560
12/11/2022, 11:13 AMA picture of the airflow exception:
c
colossal-smartphone-90274
12/12/2022, 4:09 PMI am also getting this error
b
bulky-soccer-26729
12/12/2022, 7:12 PMhey Yuval and Oliver! so for this error you posted, are you trying to delete an airflow dag / dataflow then or a specific airflow task / dataJob? and for Feast you're trying to delete feature tables?
bulky-soccer-26729
12/12/2022, 7:13 PMwe've seen some issues with our delete CLI lately and this could be part of the overall situation we're aware of. in the sort term, is it possible for you to remove all data overall and re-ingest your real data?
b
big-carpet-38439
12/12/2022, 8:16 PMCan you paste the full command you had issued as well?
c
colossal-smartphone-90274
12/13/2022, 9:18 AMI am getting the error when I try to delete a schema container that contains datasets inside it -> for example:
datahub delete --urn "urn:li:container:<container id>" --hardcolossal-smartphone-90274
12/13/2022, 9:19 AMThis error came after having the error ->
AssertionError: Did not delete all entities, try running this command again!b
brainy-piano-85560
12/13/2022, 9:22 AMHey, so I decided to just nuke datahub and re-ingest my real data.
Chris - I tried to delete Feast feature tables, and the airflow dag (also tried the tasks first). All data is from the sample-ingest.
John - I tried several (on each object that I couldn't delete). but that's for the airflow dag - datahub delete --urn "urnlidataFlow:(airflow,dag_abc,PROD)". As I said earlier, tried both soft and hard deletions
6 Views