Hi, I found out that lineage result number return is wrong between this two resolver. Mainly its because one uses

searchAcrossEntities

to check if its

soft-deleted

from

LineageSearchService.java

LineageSearchResult resultForBatch = buildLineageSearchResult(
          _searchService.searchAcrossEntities(entitiesToQuery, input, finalFilter, sortCriterion, queryFrom, querySize,
              SKIP_CACHE), urnToRelationship)

The top UI showing 9. But 2 of them are soft deleted. This one is using resolver from

EntityLineageResultResolver.java

The bottom UI show the correct value. Which is 7 Resolver from

SearchAcrossLineageResolver.java

Can you advice how to take it from here?

Hi @worried-branch-76677, our on-call for platform issues like this one is on holiday today, but will be back tomorrow, I just wanted to message to let you know we’ve logged your issue 🙂

Hello @worried-branch-76677 What version of datahub are you using?

0.9.2 . is it fixed in 0.9.3 already? @incalculable-ocean-74010

Need to check with the team, there has been some work with cache consistency on DataHub deployments that have more than 1 gms instance. Is that your case?

Not sure if its cache. Its not filtering out soft deleted at all. Maybe u can check the two resolvers?

thank you @worried-branch-76677!

Any update for this issue, are u able to replicate it?

Hello Aezo, I have not been able to replicate this issue. Today in Townhall, there will be a presentation on elastic search consistency issues that are being tackled that I think may impact this inconsistency.