fierce-garage-74290
02/01/2023, 7:07 PMdatahub ingest -c recipes/glossaries/glossary_recipe.yml I'd like to learn from the output how many definitions got actually changed. But I am afraid that whenever I run this recipe all the terms get overwritten and I am always getting in the total_records_written the number of records in the recipe.
Question: how I can determine if any glossary got modified? I need this to configure notifications for the business team (they would like to be informed whenever any glossary changes).
Thanks!bulky-soccer-26729
02/01/2023, 8:00 PMbulky-soccer-26729
02/02/2023, 5:22 PM{
"event_type": "MetadataChangeLogEvent_v1",
"event": {
"entityType": "glossaryTerm",
"entityUrn": "urn:li:glossaryTerm:ClientsAndAccounts.Testing123",
"changeType": "UPSERT",
"aspectName": "glossaryTermInfo",
"aspect": {
"value": "{\"sourceUrl\":\"<https://spec.edmcouncil.org/fibo/ontology/FBC/ProductsAndServices/ClientsAndAccounts/Balance>\",\"customProperties\":{},\"name\":\"Testing123\",\"definition\":\"Amount of money available or owed - EDITING\",\"parentNode\":\"urn:li:glossaryNode:ClientsAndAccounts\",\"sourceRef\":\"FIBO\",\"termSource\":\"EXTERNAL\"}",
"contentType": "application/json"
},
"systemMetadata": {
"lastObserved": 1675357730805,
"runId": "datahub-business-glossary-2023_02_02-12_08_50"
},
"previousAspectValue": {
"value": "{\"sourceUrl\":\"<https://spec.edmcouncil.org/fibo/ontology/FBC/ProductsAndServices/ClientsAndAccounts/Balance>\",\"customProperties\":{},\"name\":\"Testing123\",\"definition\":\"Amount of money available or owed\",\"parentNode\":\"urn:li:glossaryNode:ClientsAndAccounts\",\"sourceRef\":\"FIBO\",\"termSource\":\"EXTERNAL\"}",
"contentType": "application/json"
},
"previousSystemMetadata": {
"lastObserved": 1675357630649,
"runId": "datahub-business-glossary-2023_02_02-12_07_10"
},
"created": {
"time": 1675357730999,
"actor": "urn:li:corpuser:__datahub_system"
}
},
"meta": {
"kafka": {
"topic": "MetadataChangeLog_Versioned_v1",
"offset": 8410,
"partition": 0
}
}
}
so you could easily subscribe to specific change events and for specific entities. so if you want to see whenever a glossary term has aspects change or created. check out the docs!
https://datahubproject.io/docs/act-on-metadata
https://datahubproject.io/docs/actionsfierce-garage-74290
02/02/2023, 8:47 PMbulky-soccer-26729
02/02/2023, 8:50 PM