https://datahubproject.io logo
Join Slack
Powered by
Guys, need some help with strange lineage behavior...
# troubleshoot
c
Guys, need some help with strange lineage behavior. More info in thread
I have some dataset without usptream lineage in UI (and in API query). But i can see it in postgres. Already did elastic reindex. Do you have any ideas?
b
hey Nikita! definitely strange here. can you confirm that the dataset in the upstream lineage aspect does indeed exist?
c
hey, Chris. Yes, it exists.
@bulky-soccer-26729 hello. Do you have any idea?
b
hey Nikita - sorry I'm asking around to see if I can find an answer for you
alright question for you - what graph store are you using? elasticsearch or neo4j per chance?
c
elasticsearch
b
and can you query the graph store directly to see if this upstream edge is there?
c
Is there any info how to do that?
b
i know you said you ran restore indices above, but if it's in the DB and nothing is soft-deleted then the only thing i can think of is that the index is off for some reason
c
nothing is soft-deleted
you mean dataset in upstream lineage?
b
yeah exactly
c
soft delete change 
status
aspect to 
{"removed":true}
, right?
b
and the way to do that would be to check the graph_service_v1 index in elasticsearch and see if this edge is there. I use ElasticVue to look at elasticsearch
i would query to see if the urn in question is in there and we can see what edges exist for it
c
ok, thank you, i will try
b
the source urn should be the urn in question here (that isn't showing anything in the UI for upstream) and the destination should the the upstream urn. The relationshiptype should be 
DownstreamOf
c
I did some soft delete before, may be something went wrong
b
possibly.. but yeah soft deleing is changing the status aspect 
removed: true
like you said
c
but now it is 
removed: false
Thank you so much. I will try to find something in Elastic later.
b
yeah so now it should be good. have you run restore indices recently? possibly deleting stuff and "undeleting" them have tossed something off and a recent restore indices run won't hurt
c
have you run restore indices recently?
Yes, i do. After deleting and ingestion.
Actualy, i get this problem not first time after deleting and reingestion
2 Views
Open in Slack
PreviousNext
Powered by