Hi team, after upgrade to 0.8.16 I got the problem...
# troubleshoot
a
Hi team, after upgrade to 0.8.16 I got the problem
Copy code
POST /entities?action=searchAcrossEntities - searchAcrossEntities - 200 - 0ms
04:56:50.672 [ForkJoinPool.commonPool-worker-3] ERROR c.l.datahub.graphql.GmsGraphQLEngine - Failed to load Entities of type: DataJob, keys: [urn:li:dataJob:(urn:li:dataFlow:(airflow,ETL_30M,<http://analytics.airflows.mservice.io|analytics.airflows.mservice.io>),CORE_TRANS), urn:li:dataJob:(urn:li:dataFlow:(airflow,etl_30m,<http://analytics.airflows.mservice.io|analytics.airflows.mservice.io>),core_trans), urn:li:dataJob:(urn:li:dataFlow:(airflow,etl_6hour,<http://analytics.airflows.mservice.io|analytics.airflows.mservice.io>),core_trans), urn:li:dataJob:(urn:li:dataFlow:(airflow,ETL_30M,<http://analytics.airflows.mservice.io|analytics.airflows.mservice.io>),MERGE_CORE_TRANS), urn:li:dataJob:(urn:li:dataFlow:(airflow,etl_30m,<http://analytics.airflows.mservice.io|analytics.airflows.mservice.io>),merge_core_trans), urn:li:dataJob:(urn:li:dataFlow:(airflow,etl_6hour,<http://analytics.airflows.mservice.io|analytics.airflows.mservice.io>),merge_core_trans), urn:li:dataJob:(urn:li:dataFlow:(airflow,ETL_30M,<http://analytics.airflows.mservice.io|analytics.airflows.mservice.io>),MERGE_USERPAYMENT_SUBTRANSTYPE_CORE_TRANS), urn:li:dataJob:(urn:li:dataFlow:(airflow,etl_30m,<http://analytics.airflows.mservice.io|analytics.airflows.mservice.io>),merge_userpayment_subtranstype_core_trans), urn:li:dataJob:(urn:li:dataFlow:(airflow,etl_6hour,<http://analytics.airflows.mservice.io|analytics.airflows.mservice.io>),merge_userpayment_subtranstype_core_trans), urn:li:dataJob:(urn:li:dataFlow:(airflow,growth_daily_crm_5h_10h_12h_15h_shared,<http://analytics.airflows.mservice.io|analytics.airflows.mservice.io>),CRM_RAW_DATA_CORE_TRANS)] Failed to batch load DataJobs
04:56:50.673 [ForkJoinPool.commonPool-worker-3] ERROR c.l.d.g.e.DataHubDataFetcherExceptionHandler - Failed to execute DataFetcher
java.util.concurrent.CompletionException: java.lang.RuntimeException: Failed to retrieve entities of type DataJob
	at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
	at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1606)
	at java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1596)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
Caused by: java.lang.RuntimeException: Failed to retrieve entities of type DataJob
	at com.linkedin.datahub.graphql.GmsGraphQLEngine.lambda$null$104(GmsGraphQLEngine.java:862)
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
	... 5 common frames omitted
Caused by: java.lang.RuntimeException: Failed to batch load DataJobs
	at com.linkedin.datahub.graphql.types.datajob.DataJobType.batchLoad(DataJobType.java:106)
	at com.linkedin.datahub.graphql.GmsGraphQLEngine.lambda$null$104(GmsGraphQLEngine.java:859)
	... 6 common frames omitted
Caused by: java.lang.IllegalStateException: Duplicate key com.linkedin.metadata.entity.ebean.EbeanAspectV2@5a1a7708
g
Hey there @adamant-van-40260 - the issue has to do with the fact that you have some entities with identical names when ignoring casing but different names when taking casing into account. Right now, this case trips up Datahub. I would recommend deleting either the upper cased or lower cased version of these entities.
In the longer term, this is a problem the datahub team is looking to address.
a
thank bro, got it 😄
👍 1
Hi @green-football-43791 , can you provide the best way to delete entity ? I delete by api datahub delete --urn "...", but this record is still existed in elasticsearch 😞
g
Interesting.. that should not be happening. Are you sure you deleted the right entity? What does the document look like in elastic?
Is the entity still in mysql after the deletion?
a
sorry, the command delete entity in ES too, false alert 😄
g
ah, nice to hear!