https://datahubproject.io logo
Join Slack
Powered by
Cautionary tale with case sensitivity of urns in m...
# troubleshoot
b
Cautionary tale with case sensitivity of urns in mysql. GMS allows you to ingest multiple datajob urns that are the same (just different case), for example, 
urn:li:dataJob:(urn:li:dataFlow:(aaa,bbb,PROD),SomeName)
and 
urn:li:dataJob:(urn:li:dataFlow:(aaa,bbb,PROD),SOMENAME)
will be stored as separate rows in mysql. But when you try looking at some table in the UI that has this datajob in the downstream lineage you’re going to get an error (see thread)
Copy code
11:22:03.172 [Thread-809431] ERROR c.l.d.g.e.DataHubDataFetcherExceptionHandler:21 - Failed to execute DataFetcher
java.util.concurrent.CompletionException: java.lang.RuntimeException: Failed to retrieve entities of type DataJob
	at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
	at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1606)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Failed to retrieve entities of type DataJob
	at com.linkedin.datahub.graphql.GmsGraphQLEngine.lambda$null$144(GmsGraphQLEngine.java:1273)
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
	... 1 common frames omitted
Caused by: java.lang.RuntimeException: Failed to batch load Data Jobs
	at com.linkedin.datahub.graphql.types.datajob.DataJobType.batchLoad(DataJobType.java:118)
	at com.linkedin.datahub.graphql.GmsGraphQLEngine.lambda$null$144(GmsGraphQLEngine.java:1270)
	... 2 common frames omitted
Caused by: java.lang.IllegalStateException: Duplicate key com.linkedin.metadata.entity.ebean.EbeanAspectV2@33331602
	at java.util.stream.Collectors.lambda$throwingMerger$0(Collectors.java:133)
	at java.util.HashMap.merge(HashMap.java:1254)
	at java.util.stream.Collectors.lambda$toMap$58(Collectors.java:1320)
	at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
	at java.util.ArrayList$Itr.forEachRemaining(ArrayList.java:901)
	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
	at com.linkedin.metadata.entity.ebean.EbeanAspectDao.batchGet(EbeanAspectDao.java:227)
	at com.linkedin.metadata.entity.ebean.EbeanEntityService.getEnvelopedAspects(EbeanEntityService.java:710)
	at com.linkedin.metadata.entity.ebean.EbeanEntityService.getLatestEnvelopedAspects(EbeanEntityService.java:219)
	at com.linkedin.metadata.entity.EntityService.getEntitiesV2(EntityService.java:175)
	at com.linkedin.entity.client.JavaEntityClient.batchGetV2(JavaEntityClient.java:99)
	at com.linkedin.datahub.graphql.types.datajob.DataJobType.batchLoad(DataJobType.java:102)
	... 3 common frames omitted
b
oh gotcha very much appreciate the heads up here!
b
Yes for sure . This is a case of a bad urn getting resolved. We do need to look at this as a bug
4 Views
Open in Slack
PreviousNext
Powered by