Hi team. Good Morning. My name is Milan and I am c...
# troubleshoot
e
Hi team. Good Morning. My name is Milan and I am currently exploring the actions framework of Datahub. I have a couple of doubt regarding this. 1. Currently in the Entity Change Event v1 provides the facility for detecting any change in the schema of datasets in Datahub right? Is this limited only to Add/Remove of Schema fields (i.e. I hope that any new column inserted/removed will be captured) or can it be also extended to any kind of modification of the fields. 2. We are also planning to upload the stats for each of our dataset. Is there a way to have customised slack alerts for this. Alerts to be sent when there is some deviation in the stats or say at a certain threshold? It will be very helpful if you can kindly furnish me with these two details at the earliest
📖 1
🔍 1
l
Hey there 👋 I'm The DataHub Community Support bot. I'm here to help make sure the community can best support you with your request. Let's double check a few things first: ✅ There's a lot of good information on our docs site: www.datahubproject.io/docs, Have you searched there for a solution? ✅ button ✅ It's not uncommon that someone has run into your exact problem before in the community. Have you searched Slack for similar issues? ✅ button Did you find a solution to your issue? ❌ Sorry you weren't able to find a solution. I'm sending you some tips on info you can provide to help the community troubleshoot. Whenever you feel your issue is solved, please react ✅ to your original message to let us know!
d
@big-carpet-38439 might be able to speak to this!
e
Hi @delightful-ram-75848. Thank you for your response. @big-carpet-38439 requesting you to kindly advise me on this at the earliest. Thanks in advance.
@big-carpet-38439 Apologies for the urgency but kindly expedite your process. Awaiting your response on this. cc: @delightful-ram-75848
q
If entity_change_events doesn’t cover your use case (entity_change_events is broken in some cases in v10.1+ anyways (see issue here: https://github.com/datahub-project/datahub/issues/8007) then the meta_data_change_events should be able to cover all of your use cases! You should be able to pull information on the stats of datasets via the graphQL api on a schedule, then use the slack api to send custom alerts.
Since stats should be pulled on ingestion, you could add a transformer to your ingestion that checks the dataset stats and then sends a slack notification if they’re above a certain threshold.
e
Thank you @quiet-television-68466 for your response.
q
Happy to help if you have more questions 😄
e
Yeah @quiet-television-68466 if it is not too much to ask can I get a list of all the event categories and eventoperations for the different entities supported by Metadata Change Event Log V1 that may be specified in the action configuration file. I went through the docs but only a few them are listed there and it is also mentioned that there are many more. It would be very helpful if you can guide me in this front too
q
I haven’t been able to find great docs either, what I’ve done is used this action: https://datahubproject.io/docs/actions/actions/hello_world/ to print out to screen all the different events that I need, and create a parser to deal with those.
e
Hmm. Alright. Thank you for your time and advise.🙂