Hello! delete dataset "aspects": [...
# ingestion
c
Hello! delete dataset "aspects": [ { "com.linkedin.pegasus2avro.common.Status": { "removed": null } please help delete schema
b
Copy code
[{
    "auditHeader": null,
    "proposedSnapshot": {
        "com.linkedin.pegasus2avro.metadata.snapshot.DatasetSnapshot": {
            "urn": "urn:li:dataset:(urn:li:dataPlatform:kafka,SampleKafkaDataset,PROD)",
            "aspects": [
                {
                    "com.linkedin.pegasus2avro.common.Status": {
                        "removed": true
                    }
                }
            ]
        }
    }
}]
try this?
its not a true delete though, just remove from UI
e
^ +1
The entity page will still exist
but it will not be visible on search and browse!
b
you can still find it in UI if there is a lineage that leads to it though
e
Yes. We added a task for this! We will get to this soon. cc @green-football-43791
c
how delete schema from ui ?
e
Are you trying to update schema or delete the dataset as a whole?
c
update schema to null )
e
So basically hide the schema? Each row in the schema table is set https://github.com/linkedin/datahub/blob/master/metadata-models/src/main/pegasus/com/linkedin/schema/SchemaField.pdl#L19 https://github.com/linkedin/datahub/blob/master/metadata-models/src/main/pegasus/com/linkedin/schema/SchemaMetadata.pdl#L49 So if you set the SchemaMetadata “fields” to an empty array, you will be able to clear the schema
👍 1
a
@early-lamp-41924 Can I ask whether I can hide the whole database instead of just one dataset
e
unfortunately removing based on search (i.e. search for datasets in the db) is not supported right now. For now, you could perform a search request to gms and generate the above mce (for setting removed to true) for each dataset in the search response
b
on this topic, sorry to side-track but search request to GMS only returns version 0 of each dataset in MySQL right?
e
Search in gms as of now returns urns that match the query
the fields in the index are based on the latest version of each aspect
since an update to the aspect triggers a MetadataAuditEvent which updates the search and graph indices
b
cool, thanks. was thinking of a side project that explores the past changes to datasets. but seems to me that i should query the MySQL database directly
e
we have a PR out that adds historical versions of entities to the UI! If that is what you are looking for
b
sounds relevant! but i dont see it in the PR list now?
e
actually realized its merged
take a look at this end point!
b
its currently exists as endpoint only right? the UI will be updated to to utilise it?
e
yes! cc @green-football-43791
g
Yes- work is currently in progress to have the UI read historical schemas
we look forward to demoing it for you soon!
❤️ 2
a
@green-football-43791 @early-lamp-41924 Seems like I still can get some hints for deleted datasets
Anyway, can we have a button on UI to delete dataset directly without running script?
👍 1
c
Hung Nguyen are you not like scripts? ) i think "auditHeader": null, "proposedSnapshot": { "com.linkedin.pegasus2avro.metadata.snapshot.DatasetSnapshot": { i agree that it 's rather difficult to understand and use)