https://datahubproject.io logo
Join Slack
Powered by
Hello, i'm testing out datahub actions framework w...
# troubleshoot
n
Hello, i'm testing out datahub actions framework with v0.8.35. i started the hello world consumer and runned some ingestions and performed actions on ui. Cannot figure out why i'm receiving only MetadataChangeLog_v1 events and no EntityChangeEvent_v1 at all. setup jobs for kafka runned correctly on our helm deployment. any idea?
e
Sorry about the late reply. Which version are you running for the hello world consumer / gms?
n
no worries, i'm running datahub gms 0.8.35 and the consumer is a separate docker container with the following requirements acryl-datahub==0.8.35.1 acryl-datahub-actions==0.0.1
e
Can you share the env variables sent when deploying gms?
n
we are not overriding or adding any new env variable to gms deployment. but we are using a local copy of the helm chart in our gitops repo. I'm thinking that it's possible maybe we missed some update of the chart that is needed. we probably have updated only the image version tag before deploying. are you thinking at something in particular that should be setted and i can check? otherwise i will try to sync all the chart with the up to date version and deploy again
e
cc @big-carpet-38439 are there any envs that needs to be set to get the EntityChangeEvent_v1 events?
b
PE_CONSUMER_ENABLED=true in either GMS or the standalone event consumer!
But I believe this should be part of the charts by default - let me verify
e
Yeah. Seems like Francesco is using an old chart version, which missed this env variable
b
I think we missed the helm changes here. We'll be sure to add that. For now, simply add the following the "extraEnvs" section of your values.yaml block for `datahub-gms`:
Copy code
- name: PE_CONSUMER_ENABLED
              value: "true"
n
thanks for the suggestion guys, i tried adding the variable both on gms and on mae consumer with standalone disabled, but i'm still getting only MetadataChangeLogEvent_v1 on my action consumer container
i plugged a kafka console consumer to topic PlatformEvent_v1 and i'm not receiving anything. so probably the issue is upstream in the publisher of this event. if you have any other advice on it, i will test it 🙂
b
hm
let me take another look. MAE consumer is also up right?
So any changes to an entity should be resulting in an EntityChangeEvent.. very surprised this isn't occurring
Even on 0.8.35 gms
n
yes MAE is running with PE consumer enabled, from my last update i also synced our helm chart with all the latest changes of your latest version. but still getting nothing
oh forgot to say, i updated to 0.8.38 also
Hey, i'm back working on this. PlatformEvent_v1 topic is alway empty for me. i saw inside the pod that my enityt-registry.yml has no events declared, even if it's the same also on datahub repo. i tried to mount a entity yaml with declared
Copy code
events:
  - name: entityChangeEvent
but still not working. tested with latest datahub version anyone that can help on it?
r
Hey there! 👋 Make sure your message includes the following information if relevant, so we can help more effectively! 1. Which DataHub version are you using? (e.g. 0.12.0) 2. Please post any relevant error logs on the thread!
t
@numerous-application-54063 were you able to solve this? We're also not seeing any EntityChangeEvent_v1 events. We're using Datahub 0.12.0.
r
Those messages are part of platform events. No change is needed to the entity registry. If running standalone consumers, the MAE consumer should have this environment variable set to `true`⁣ , or GMS if not running standalone: https://github.com/datahub-project/datahub/blob/11f7804b1ea2f47495c336e8d91d7dc380ccd228/metadata-jobs/pe-consumer/src/main/java/com/datahub/event/PlatformEventProcessorCondition.java#L12
t
Hi @brainy-tent-14503, thanks for your reply. We're running standalone consumers and PE_CONSUMER_ENABLED is set to true on the MAE consumer. However, it seems that the messages are not even produced for the platform event topic. Messages for other topics e.g. the metadata change log topic are produced and consumed successfully. I can see debug logs messages in GMS about the 
KafkaEventProducer.java
producing MetadataChangeLogs but there's nothing about platform events (e.g. method https://github.com/datahub-project/datahub/blob/e6d706687ab5af6b888176bb775c338ff9[…]java/com/linkedin/metadata/dao/producer/KafkaEventProducer.java). To me it feels like this started to happen when we switched to standalone consumers but I could be wrong. Topics and consumer groups are set up correctly and the ENV vars referring to the topics also seem to be correct. Any idea?
So I just switched back to integrated consumers and I see the EntityChangeEvent_v1 events being triggered again. Changing back to standalone consumers by setting 
datahub_standalone_consumers_enabled
to true in the Helm values brings back the issue. @brainy-tent-14503 could there be an issue or are we missing something? We're using the datahub Helm chart version 0.3.8 and Datahub 0.12.0.
r
Do you see the environment variable on the mae consumer pod? Specifically the one here: https://github.com/acryldata/datahub-helm/blob/master/charts/datahub/subcharts/datahub-mae-consumer/templates/deployment.yaml#L114
Also if you could share the logs from MAE consumer, it might have some log statements.
t
Hi @brainy-tent-14503, yes I see the ENV vars on the pod (see screenshot). I'll send you some logs via DM. Many thanks for your support.
r
I do see the expected hook being initialized in the mae consumer when that env is enabled, however I am also not seeing messages being generated, will need to investigate further
👍 1
t
Thanks for looking into it.
Hi @brainy-tent-14503 any updates on this?
@brainy-tent-14503 let me know if I can support somehow.
7 Views
Open in Slack
PreviousNext
Powered by