https://datahubproject.io logo
#troubleshoot
Title
# troubleshoot
b

bumpy-activity-74405

06/15/2022, 10:00 AM
Cautionary tale with case sensitivity of urns in mysql. GMS allows you to ingest multiple datajob urns that are the same (just different case), for example,
urn:li:dataJob:(urn:li:dataFlow:(aaa,bbb,PROD),SomeName)
and
urn:li:dataJob:(urn:li:dataFlow:(aaa,bbb,PROD),SOMENAME)
will be stored as separate rows in mysql. But when you try looking at some table in the UI that has this datajob in the downstream lineage you’re going to get an error (see thread)
Copy code
11:22:03.172 [Thread-809431] ERROR c.l.d.g.e.DataHubDataFetcherExceptionHandler:21 - Failed to execute DataFetcher
java.util.concurrent.CompletionException: java.lang.RuntimeException: Failed to retrieve entities of type DataJob
	at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
	at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1606)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Failed to retrieve entities of type DataJob
	at com.linkedin.datahub.graphql.GmsGraphQLEngine.lambda$null$144(GmsGraphQLEngine.java:1273)
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
	... 1 common frames omitted
Caused by: java.lang.RuntimeException: Failed to batch load Data Jobs
	at com.linkedin.datahub.graphql.types.datajob.DataJobType.batchLoad(DataJobType.java:118)
	at com.linkedin.datahub.graphql.GmsGraphQLEngine.lambda$null$144(GmsGraphQLEngine.java:1270)
	... 2 common frames omitted
Caused by: java.lang.IllegalStateException: Duplicate key com.linkedin.metadata.entity.ebean.EbeanAspectV2@33331602
	at java.util.stream.Collectors.lambda$throwingMerger$0(Collectors.java:133)
	at java.util.HashMap.merge(HashMap.java:1254)
	at java.util.stream.Collectors.lambda$toMap$58(Collectors.java:1320)
	at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
	at java.util.ArrayList$Itr.forEachRemaining(ArrayList.java:901)
	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
	at com.linkedin.metadata.entity.ebean.EbeanAspectDao.batchGet(EbeanAspectDao.java:227)
	at com.linkedin.metadata.entity.ebean.EbeanEntityService.getEnvelopedAspects(EbeanEntityService.java:710)
	at com.linkedin.metadata.entity.ebean.EbeanEntityService.getLatestEnvelopedAspects(EbeanEntityService.java:219)
	at com.linkedin.metadata.entity.EntityService.getEntitiesV2(EntityService.java:175)
	at com.linkedin.entity.client.JavaEntityClient.batchGetV2(JavaEntityClient.java:99)
	at com.linkedin.datahub.graphql.types.datajob.DataJobType.batchLoad(DataJobType.java:102)
	... 3 common frames omitted
b

bulky-soccer-26729

06/15/2022, 4:00 PM
oh gotcha very much appreciate the heads up here!
b

big-carpet-38439

07/20/2022, 10:26 PM
Yes for sure . This is a case of a bad urn getting resolved. We do need to look at this as a bug
4 Views